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Why GAO Did This Study 

Customs and Border Protection's 
(CBP) Container Security Initiative 
(CSI) aims to identify and examine 
high-risk U.S.-bound cargo at 
foreign seaports. GAO reported in 
2003 and 2005 that CSI helped to 
enhance homeland security, and 
recommended actions to 
strengthen the program. This 
report updates information and 
assesses how CBP has (1) 
contributed to strategic planning 
for supply chain security, (2) 
strengthened CSI operations, and 
(3) evaluated CSI operations. To 
address these issues, GAO 
interviewed CBP officials and 
reviewed CSI evaluations and 
performance measures. GAO also 
visited selected U.S. and CSI 
seaports, and met with U.S. and 
foreign government officials. 



What GAO Recommends 



GAO recommends that CBP 
enhance data collected on CSI 
team performance, host 
government examinations, and 
related performance measures. 
CBP concurred with the recom- 
mendation to enhance data on 
team performance. It partially 
concurred with the need to 
enhance data on host examin- 
ations, stating that it already 
conducts actions to improve such 
data. However, these actions do 
not systematically collect data on 
people, processes, or technology 
used by host governments to 
examine U.S.-bound containers. 
CBP partially concurred with the 
need to enhance performance 
measures, but stated it already 
captures core program functions. 
We still see room for improvement. 

To view the full product, including the scope 
and methodology, click on GAO-08-187. 
For more information, contact Stephen L. 
Caldwell at (202) 512-9610 or 
caldwells @ gao.gov. 



What GAO Found 

By collaborating on the development of the Department of Homeland 
Security's Strategy to Enhance International Supply Chain Security, and by 
revising the CSI strategic plan as GAO recommended, CBP has contributed to 
the overall U.S. strategic planning efforts related to enhancing the security for 
the overseas supply chain. Also, CBP reached its targets of operating CSI in 58 
foreign seaports, and thereby having 86 percent of all U.S.-bound cargo 
containers pass through CSI seaports in fiscal year 2007 — representing a 
steady increase in these measures of CSI performance. 

To strengthen CSI operations, CBP has sought to address human capital 
challenges and previous GAO recommendations by increasing CSI staffing 
levels closer to those called for in its staffing model and revising its human 
capital plan. However, challenges remain because CBP continues to rely, in 
part, on a temporary workforce; has not determined how to optimize its 
staffing resources; and reports difficulties in identifying sufficient numbers of 
qualified staff. In addition, CBP has enhanced relationships with host 
governments participating in CSI. However, hurdles to cooperation remain at 
some seaports, such as restrictions on CSI teams witnessing examinations. 

CBP improved its evaluation of CSI team performance at seaports, but 
limitations remain in the evaluation process that affect the accuracy and 
completeness of data collected. CBP has not set minimum technical criteria 
for equipment or systematically collected information on the equipment, 
people, and processes involved in CSI host government examinations of high- 
risk, U.S-bound container cargo. Also, CBP has not developed general 
guidelines to use in assessing the reliability of these examinations. Thus, CBP 
potentially lacks information to ensure that host government examinations 
can detect and identify weapons of mass destruction, which is important 
because containers are typically not reexamined in the United States if 
already examined at a CSI seaport. CBP refined overall CSI performance 
measures, but has not fully developed performance measures and annual 
targets for core CSI functions, such as the examination of high-risk containers 
before they are placed on vessels bound for the United States. These 
weaknesses in CBP's data collection and performance measures potentially 
limit the information available on overall CSI effectiveness. 



Containers Stacked on a Vessel at a CSI Port 




Source: GAO. 
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Oceangoing cargo containers play a vital role in the movement of cargo 
between global trading partners. In fiscal year 2007, more than 10 million 
oceangoing cargo containers arrived at U.S. seaports — meaning roughly 
28,000 oceangoing containers arrived each day that year. The terrorist 
attacks of 2001 heightened concerns about the ability of both the federal 
government and companies participating in international maritime 
commerce to identify and prevent weapons of mass destruction from 
being smuggled inside cargo containers bound for the United States. 
Balancing security concerns with the need to facilitate the free flow of 
commerce remains an ongoing challenge for the public and private sectors 
alike. 

In the federal government, U.S. Customs and Border Protection (CBP), 
part of the Department of Homeland Security (DHS), is charged with 
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managing, securing, and controlling the nation's border and in its capacity 
as the frontline border security agency, plays a lead role in facing maritime 
threats. CBP launched the Container Security Initiative (CSI) in January 
2002, which through partnerships with its foreign counterparts, is designed 
to help protect global trade lanes by targeting and examining container 
cargo that poses a threat as early as possible in the global supply chain. As 
part of the program, foreign governments allow CBP officers to be 
stationed at foreign seaports. These officers use intelligence and 
automated risk assessment information to target shipments to identify 
those at risk of containing weapons of mass destruction (WMD) or other 
terrorist contraband. CBP and host government officials share the role of 
assessing the risk of U.S.-bound container cargo leaving the seaports of 
countries participating in CSI. CBP officers at the CSI seaports are 
responsible for targeting high-risk cargo shipped in containers and other 
tasks, whereas host government customs officials examine the high-risk 
cargo — when requested by CBP — by scanning containers using various 
types of nonintrusive inspection equipment, such as large-scale X-ray 
machines, or by physically searching the container's contents before it 
travels to the United States. 

As part of its strategic plan, CBP is partnering with international trade and 
security groups to develop supply chain security standards that can be 
implemented by the world community. By engaging international 
organizations, CBP is contributing to the development of global security 
standards. Recent legislative actions intended to further enhance 
maritime security also updated requirements that affect CSI. In October 
2006, Congress passed and the President signed legislation — the Security 
and Accountability for Every Port Act (SAFE Port Act) 1 — establishing a 
statutory framework for CSI, which previously had been an agency 
initiative not specifically required by law. The act imposed various 
mandates, such as requiring CBP to take risk factors including cargo 
volume into account when designating seaports as CSI participants. In 
August 2007, the Implementing Recommendations of the 9/11 Commission 
Act of 2007 (9/11 Act) was enacted, which requires, among other things, 
100 percent scanning of U.S.-bound cargo containers by foreign seaports 



1 Pub. L. No. 109-347, 120 Stat. 1884 (2006). 
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by 2012, with possible extensions for some ports — replacing a similar 
provision in the SAFE Port Act that did not have a deadline. 2 

We have previously reported on CSFs progress in meeting its strategic 
goals and objectives. Our July 2003 and April 2005 reports on CSI 
acknowledged the program's important role in helping to enhance 
homeland security, but we also recommended actions to enhance the 
strategic planning for the program, such as better defining its goals, 
objectives, and performance measures. In addition, we recommended 
actions to strengthen the program's management and operations, such as 
conducting human capital planning (which affects CBP staffing levels at 
seaports) and establishing minimum technical capability requirements for 
equipment used to examine high-risk containers. 3 

Recognizing the importance of the CSI program, you asked us to conduct 
another review. For this report, we assessed the following issues: 

• How has CBP contributed to strategic planning for supply chain 
security efforts and the CSI program in particular, and what progress 
has been made in achieving CSI performance goals? 

• How has CBP strengthened CSI operations in response to our 2005 
review and what challenges, if any, remain? 

• How does CBP evaluate CSI seaport operations and assess program 
performance overall, and how has this process changed over time? 

To address these objectives, we met with CBP officials who have program 
responsibilities for CSI, and reviewed available program data and 
documentation. Specifically, to review CBP's strategic planning initiatives, 
we reviewed national-level strategic planning documents and those 
created for DHS, CBP, and CSI. To determine CBP's progress in achieving 



2 Pub. L. No. 110-53, § 1701(a) 121 Stat. 266, 489-90 (2007) (amending 6 U.S.C. § 982(b)). 
Also, see GAO, Maritime Security: Maritime Security: The SAFE Port Act: Status and 
Implementation One Year Later. GAO-08-126T. (Washington, D.C.: Oct. 30, 2007), mainly 
pages 31 to 35, and 44 to 48 for more detail on the requirements included in the SAFE Port 
and 9/11 Acts that affect the CSI program. 

3 GAO, Container Security: A Flexible Staffing Model and Minimum Equipment 
Requirements Would Improve Overseas Targeting and Inspection Efforts, GAO-05-557 
(Washington, D.C.: Apr. 26, 2005), and GAO, Container Security: Expansion of Key 
Customs Programs Will Require Greater Attention to Critical Success Factors, 
GAO-03-770 (Washington, D.C.: July 25, 2003). 
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its CSI program goals, we reviewed CBP's statistical data on container 
cargo and CSI program activities. To learn about how CBP has 
strengthened its operations, we reviewed our previous assessments of the 
CSI program, and examined CBP's efforts to implement our three prior 
recommendations. To assess CBP's progress, we met with CBP officials at 
the U.S. National Targeting Center - Cargo (NTCC) in Virginia and three 
domestic seaports in different geographical locations and representing 
varying volumes of container traffic. 4 We also visited six CSI seaports 
located overseas that were selected based on several factors, including 
geographic and strategic significance, volume of container traffic, and 
when CSI operations began at the seaport. The results from our visits to 
seaports provided examples of CBP and host government operations but 
cannot be generalized beyond the seaports visited because we did not use 
statistical sampling techniques in selecting the seaports. 

To determine what progress CBP has made in strengthening its tools for 
monitoring and measuring the progress of the CSI program, we reviewed 
the performance measures presented in the CSI strategic plan against 
criteria developed by the Office of Management and Budget and GAO. We 
also reviewed a nonrepresentative sample of CSI team evaluations. While 
these documents provided examples about program evaluation methods 
and CSI program operations, and generally corroborated our seaport site 
visit observations, our findings cannot be generalized to the program as a 
whole. We conducted this performance audit from May 2006 through 
January 2008 in accordance with generally accepted government auditing 
standards. Those standards require that we plan and perform the audit to 
obtain sufficient, appropriate evidence to provide a reasonable basis for 
our findings and conclusions based on our audit objectives. We believe 
that the evidence obtained provides a reasonable basis for our findings 
and conclusions based on our audit objectives. A detailed discussion of 
our scope and methodology is contained in appendix I. 



4 At the time of our engagement, we met with officials from the NTC. The NTC was 
established in response to the need for proactive targeting aimed to prevent acts of terror 
and to seize, deter and disrupt terrorists and/or implements of terror. The NTC originally 
combined both passenger and cargo targeting in one facility. It was later divided into the 
National Targeting Center - Cargo (NTCC) and the National Targeting Center - Passenger 
(NTCP). By convention, we will use NTCC in our references since its mission is to support 
CBP cargo targeting operations. 
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RGSllltS in Brief CBP c ontr ibut e d to a new strategic planning document to guide efforts to 

lj secure the international supply chain, updated the CSI strategic plan, and 

reported achieving key CSI goals by increasing both the number of CSI 
locations and the proportion of total U.S.-bound containers passing 
through CSI seaports. By supporting the development of the Strategy to 
Enhance International Supply Chain Security that DHS was required to 
produce in accordance with the SAFE Port Act, and by revising and 
enhancing the strategic plan for CSI, as we had previously recommended, 
CBP has contributed to overall strategic planning efforts related to supply 
chain security. The supply chain security strategy, issued in July 2007, 
delineates the supply chain security roles, responsibilities, and authorities 
of federal, state, local, and private sector entities, and describes how 
CBP's portfolio of initiatives to address supply chain security, including 
CSI, is coordinated throughout the supply chain. As to achieving CSI 
performance goals, in fiscal year 2007, CBP reached its targets of 
operating CSI in 58 foreign seaports and having 86 percent of all U.S.- 
bound containers passing through CSI seaports — where, according to 
CBP, there is an opportunity for the high-risk cargo to be examined at 
foreign seaports before reaching the United States. These results represent 
a continued increase in both of these measures of CSI performance since 
CSI's inception in 2002. Also, CBP reported increases in the number of 
high-risk cargo containers examined by host governments at CSI seaports. 

To strengthen CSI operations over the last 2 years, CBP has sought to 
address human capital challenges and enhance relationships with host 
governments participating in CSI, but operational challenges remain. Our 
2005 CSI report noted that CBP had not achieved its goal of targeting all 
U.S.-bound containers passing through CSI seaports for high-risk cargo 
before they depart for the United States because, in part, the agency had 
not been able to place enough staff at some CSI seaports. CBP has 
subsequently taken steps to implement related recommendations, 
including increasing overall staffing levels just above the 203 positions 
called for in its staffing allocation model by, among other things, adding 
125 permanent staff to CSI seaports and 15 staff to the NTCC, resulting in 
a parallel increase in the volume of container cargo that is targeted. 
Nevertheless, CBP continues to rely, in part, on a temporary workforce at 
CSI seaports and the NTCC; has yet to determine how to optimize its 
staffing resources even as the CSI program expands; and reports 
difficulties in identifying sufficient numbers of qualified individuals to hire 
for the program. In addition, findings from our CSI port site visits and our 
review of select CSI evaluations conducted by CBP suggest that 
relationships with host governments have improved over time, leading to 
increased information sharing between governments and a bolstering of 
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host government customs and port security practices, among other things. 
However, we also found that levels of collaboration between U.S. and host 
government officials varied across CSI seaports and we identified hurdles 
to cooperation between CSI teams and their counterparts in the host 
government, such as host country legal restrictions that CBP officials said 
prevent CSI teams from observing examinations. CBP also continues to 
face logistical difficulties inherent in a seaport environment, often outside 
of its control, such as high-risk container cargo that is infeasible to access 
for examination. 

CBP has enhanced how it collects data about CSI operations by 
strengthening its approach to on-site evaluations of teams of CBP officers 
at CSI seaports and has refined certain programwide performance 
measures, but weaknesses remain in CBP's evaluation and performance 
measurement efforts. Also, CBP lacks a process for systematically 
gathering information on the equipment, people, and processes used by 
host governments to examine U.S.-bound cargo containers identified as 
high-risk. Specifically, CBP has recently improved its process for 
conducting evaluations at CSI seaports by, among other things, testing the 
proficiency of the CBP officers who must identify high-risk cargo, and by 
introducing an electronic tool that enables CBP evaluation teams to 
systematically record their evaluative data. However, evaluators do not 
always use the data collection tools as intended, which makes it difficult 
for CBP to ensure that evaluations are consistently carried out or that 
evaluative data are reliable for management decisions. Also evaluators do 
not always follow up on recommendations made in previous evaluation 
reports — so CBP cannot ensure that previously identified problems have 
been addressed. Also, CBP has not set minimum technical capability 
criteria for equipment used at CSI seaports as we recommended in April 
2005 and as required under the SAFE Port Act for CSI and the 9/11 Act for 
future 100 percent scanning of U.S.-bound containers. Without such 
criteria and systematically collected information on equipment, people and 
processes involved in each host government's cargo, CBP does not have a 
sound basis for determining the reliability of the examination systems 
used at CSI seaports, which is of particular importance because only a 
small fraction of U.S.-bound high-risk containers are reexamined upon 
arrival in the United States. With respect to assessing CSI performance 
overall, over the past 2 years, CBP has revised its performance 
measurement system to provide decision makers with more accurate 
indicators of the program's progress — by setting some specific 
performance targets and modifying some existing measures — and to 
reflect CSI's continuing maturation. However, we identified limitations 
with the CSI performance measures, such as the omission of measures for 
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a key core CSI function, the lack of annual performance targets, and 
misleading or confusing methods for calculating several performance 
measures. These limitations may potentially make it difficult for CBP and 
DHS managers and Congress to appropriately provide program oversight. 

We are recommending that the Secretary of Homeland Security direct the 
Commissioner of CBP to take actions to help ensure that the agency has 
the information necessary to ensure that CSI is operating efficiently and 
effectively. Specifically, we are recommending that CBP (1) strengthen it's 
process for evaluating CSI teams at overseas ports by maintaining 
evaluation data, ensuring evaluation teams follow procedures, and 
monitoring the completions of recommendations from previous 
evaluations; (2) improve, in collaboration with host government officials, 
the information gathered about the host governments' examination 
systems to determine their reliability and whether mitigating actions or 
incentives are necessary to provide the desired level of security; and (3) 
enhance CSI performance measurement processes to better assess CSI 
performance overall. 

We provided a draft of this report to the Department of State and DHS for 
their review and comment. The Department of State did not provide 
written comments but provided technical comments, which have been 
incorporated into the report as appropriate. DHS provided written 
comments — incorporating comments from CBP — on December 20, 2007, 
which are presented in Appendix II. In its written comments, DHS and 
CBP concurred with our recommendation on strengthening its process for 
evaluating CSI teams at overseas locations. CBP partially concurred with 
our recommendation to improve information gathered about host 
government examination systems. CBP agreed on the importance of an 
accepted examination process and noted it continues to take steps to 
address improvements in the information gathered about host 
governments' examination systems at CSI ports by working directly with 
host government counterparts, through the World Customs Organization, 
and providing capacity building training and technical assistance. While 
CBP does engage in capacity building with some CSI host governments, it 
does not systematically collect or assess information on the people, 
processes, or technology used by these host governments to examine high- 
risk U.S.-bound containers, and thus has limited assurance that CSI host 
government examination systems can detect and identify WMD. Finally, 
CBP partially concurred with our recommendation to enhance CSI 
performance measures to better assess CSI performance overall. CBP 
stated that it believes its current measures address core program functions 
of targeting and collaboration with host governments to mitigate or 
substantiate the risk of a maritime container destined for the United 
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States. We disagree. As discussed in this report, a core element of the CSI 
program, specifically the extent to which U.S.-bound containers carrying 
high-risk cargo are examined at CSI seaports, is not addressed through 
CBP's performance measures. 



Background 



Seaports are critical gateways for the movement of commerce through the 
international supply chain. The facilities, vessels, and infrastructure within 
seaports, and the cargo passing through them, all have vulnerabilities that 
terrorists could exploit. The containers carrying goods that are shipped in 
oceangoing vessels are of particular concern because they can be filled 
overseas at many different locations and are transported through complex 
logistics networks before reaching U.S. seaports. 

In addition, transporting such a shipping container from its international 
point of origin to its final destination involves many different participants 
and many points of transfer. The materials in a container can be affected 
not only by the manufacturer or supplier of the material being shipped, but 
also by carriers who are responsible for getting the material to a port and 
by personnel who load containers onto the ships. Others who interact with 
the cargo or have access to the records of the goods being shipped 
include, among others, exporters who make arrangements for shipping 
and loading, freight consolidators who package disparate cargo into 
containers, and forwarders who manage and process the information 
about what is being loaded onto the ship. Figure 1 illustrates many of the 
key participants and points of transfer involved from the time that a 
container is loaded for shipping to its arrival at the destination seaport and 
ultimately the importer. 



Vulnerabilities of 
Containers in the 
International Supply Chain 
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Figure 1: Overview of Key Participants Involved in Shipping Containers in the International Supply Chain 
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Source: GAO, DHS. 



Several studies on maritime security conducted by federal, academic, 
nonprofit, and business organizations have concluded that the movement 
of oceangoing cargo in containers is vulnerable to some form of terrorist 
action, largely because of the movement of cargo throughout the supply 
chain. Every time responsibility for cargo in containers changes hands 
along the supply chain there is the potential for a security breach, and 
thus, vulnerabilities exist that terrorists could take advantage of by placing 
a WMD into a container for shipment to the United States. While there 
have been no known incidents of containers being used to transport 
WMDs, criminals have exploited containers for other illegal purposes, 
such as smuggling weapons, people, and illicit substances, according to 
CBP officials. Finally, while CBP has noted that the likelihood of terrorists 
smuggling WMD into the United States in cargo containers is low, the 
nation's vulnerability to this activity and the consequences of such an 
attack are potentially high. In 2002, Booz Allen Hamilton sponsored a 
simulated scenario in which the detonation of weapons hidden in cargo 
containers shut down all U.S. seaports over a period of 12 days. The 
results of the simulation estimated that the port closure could result in a 
loss of $58 billion in revenue to the U. S. economy, along with significant 
disruptions to the movement of trade. 
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Efforts to Secure 
Containers in the 
International Supply Chain 



The federal government has taken many steps to secure the supply chain, 
including the cargo in containers destined for the United States. While 
CBP officials at domestic seaports continue efforts to identify and 
examine high-risk imports arriving in containers, CBP's post-September 11 
strategy also involves focusing security efforts beyond U.S. borders to 
target and examine high-risk cargo before it enters U.S. seaports. CBP's 
strategy is based on a layered approach of related initiatives that attempt 
to focus resources on potentially risky cargo shipped in containers while 
allowing other containers carrying cargo to proceed without unduly 
disrupting commerce into the United States. CBP has initiated most of 
these efforts, shown in table 1. However, the Department of Energy (DOE) 
has led U.S. efforts to detect radiation in cargo containers originating at 
foreign seaports. 
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Table 1 : Major U. S. Initiatives to Secure Oceangoing Containers 



Initiative and year introduced Department Description 

Automated Targeting DHS CBP uses this computerized decision support tool to review documentation, including electronic 

System, (ATS), 1 995 manifest information submitted by ocean carriers on all cargo destined for the United States to 

(prototype) help identify shipments requiring additional scrutiny. ATS utilizes complex mathematical models 

with weighted rules that assign a risk score to each shipment based on manifested information. 
CBP officers review the rule firings that support the ATS score to help them make decisions on 
the extent of documentary review or examination to be conducted. 

24-hour rule, 2002 DHS CBP generally requires ocean carriers to electronically transmit cargo manifests to 

CBP's Automated Manifest System 24 hours before the U.S. -bound cargo is loaded 
onto a vessel at a foreign seaport. Carriers and importers are to provide information to 
CBP that is used to strengthen how ATS assigns risk scores. The cargo manifest 
information is submitted by ocean carriers on all arriving cargo shipments, and entry 
data (more detailed information about the cargo) are submitted by brokers. 

CSI places staff at participating foreign seaports to work with host country customs 
officials to target and examine high-risk cargo to be shipped in containers for weapons 
of mass destruction before they are shipped to the United States. CBP officials identify 
the high-risk containers and request that their foreign counterparts examine the contents 
of the containers. 



Container Security Initiative DHS 
(CSI), 2002 



Customs-Trade Partnership DHS CBP develops voluntary partnerships with members of the international trade 

Against Terrorism (C-TPAT), community comprised of importers; customs brokers; forwarders; air, sea, and land 

2002 carriers; and contract logistics providers. Private companies agree to improve the 

security of their supply chains in return for various benefits, such as a reduced likelihood 

that their containers will be examined. 

Megaports Initiative, 2003 DOE DOE installs radiation detection equipment at key foreign seaports, enabling foreign 

government personnel to use radiation detection equipment to screen shipping 
containers entering and leaving these seaports, regardless of the containers' 
destination, for nuclear and other radioactive material that could be used against the 
United States and its allies. 



Secure Freight Initiative, DHS, DOE Pilot program at selected CSI seaports to scan 100 percent of U.S. -bound cargo 
2007 containers for nuclear and radiological materials overseas using integrated examination 

systems that couple nonintrusive inspection equipment and radiation detection 

equipment. 

Source: GAO. 

Note: Cargo manifests are prepared by the ocean carrier and are composed of bills of lading for each 
shipment of cargo loaded on a vessel to describe the contents of the shipments. The bill of lading 
includes a variety of other information, such as the manufacturer of the cargo and the shipping line. 



In January 2002, CBP began CSI to target container cargo at overseas 
seaports so that high-risk cargo could be examined prior to departure for 
the United States. More recently, Congress passed legislation affecting the 
CSI program, including (1) the SAFE Port Act enacted in October 2006 that 
established a statutory framework for CSI and, among other things, 
required a pilot program, now known as the Secure Freight Initiative, to 
determine the feasibility of 100 percent scanning of U.S.-bound cargo 
containers at foreign seaports; and (2) the 9/11 Act enacted in August 2007, 
that, among other things, requires by 2012, the scanning of all U.S.-bound 
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containers at foreign seaports with potential exceptions if a seaport 
cannot meet that deadline. 

For the CSI program, CBP officials stated that DHS expended about $138 
million and $143 million, respectively for fiscal years 2006 and 2007. The 
President's budget for fiscal year 2008 requested $156 million for CSI. CSI 
is now operating at 58 seaports in 33 foreign countries, as shown in figure 
2. Appendix III lists the specific CSI seaports. 



Page 12 



GAO-08-187 Supply Chain Security 



Figure 2: Map of World with Countries Participating in CSI 
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Source: GAO (map art), Map Resources (map), CBP (data). 



Core Elements and 
Security Activities of CSI 
Program 



According to CBP, the three core elements of CSI include (1) CBP 
identifying high-risk containers; (2) CBP requesting, where necessary, that 
host governments examine high-risk containers before they are shipped; 
and (3) host governments conducting examinations of high-risk 
containers. To integrate these elements into CSI operations, CBP 
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negotiated and entered into bilateral, nonbinding arrangements with 
foreign governments, specifying the placement of CBP officials at foreign 
seaports and the exchange of information between CBP and foreign 
customs administrations. To participate in CSI, a host nation must meet 
several criteria developed by CBP. The host nation must utilize (a) a 
seaport that has regular, direct, and substantial container traffic to 
seaports in the United States; (b) customs staff with the capability of 
examining cargo originating in or transiting through its country; and (c) 
nonintrusive inspection equipment with gamma or X-ray capabilities and 
radiation detection equipment. Additionally, each potential CSI port must 
indicate a commitment to (d) establish an automated risk management 
system for identifying potentially high-risk container cargo; (e) share 
critical data, intelligence, and risk management information with CBP 
officials; (f) conduct a seaport assessment to ascertain vulnerable links in 
a port's infrastructure and commit to resolving those vulnerabilities; and 
(g) maintain a program to prevent, identify, and combat breaches in 
employee integrity. 

As part of the arrangements with foreign governments participating in CSI, 
CBP most often stations teams of CBP officers at each foreign seaport to 
conduct CSI activities in collaboration with host government customs 
officials. While the number of CBP officers stationed at CSI seaports varies 
by location, typically a CSI team consists of (1) a CSI team leader, who 
manages the team and monitors the relationship with the host country; (2) 
CBP officers, who target high-risk cargo and observe (where possible) the 
host government's examination of containers carrying the cargo; (3) an 
intelligence research specialist, who assimilates data to support timely and 
accurate targeting of containers; and (4) a special agent responsible for 
CSI-related investigations at the seaport. According to CBP, it is ideal for 
the CSI team to be located in close physical proximity with host 
government customs counterparts to facilitate collaboration and 
information sharing. However, CBP officials also stated that the agency 
uses CBP officers stationed at the NTCC as needed to support the CBP 
officers located at the CSI seaports. The CBP officials at NTCC assist the 
CSI teams at high-volume seaports to ensure all containers that pass 
through CSI seaports are targeted to identify high-risk container cargo; 
carry out CSI targeting responsibilities for CSI seaports that do not have 
CBP officials stationed there; and, according to CBP officials, conduct 
targeting for U.S.-bound container cargo that does not pass through CSI 
seaports using national sweeps to identify high-risk container cargo. 
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At CSI seaports, CBP officers share responsibilities with host 
governments' customs officials to target and examine high-risk container 
cargo. Figure 3 describes the activities carried out by CBP officers and 
host government customs officials, respectively, to target and examine 
high-risk container cargo at CSI seaports. 5 



5 We selected the terms for this report based on definitions provided in the SAFE Port Act 
and in consultation with CBP. Also, see appendix FV for more details about these CSI 
activities. 
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Figure 3: CSI Targeting and Examination Activities 



Targeting high-risk container shipments. 

CBP uses ATS to electronically review data about 
U.S. -bound shipments to produce a risk score, a 
process CBP refers to as screening. CBP officers 
review the ATS risk scores and may consider additional 
information or collaborate with host government officials 
to identify high-risk shipments with a nexus to 
terrorism — a process referred to as targeting. 
CBP officials make a final determination about which 
containers are high risk and will be referred to host 
government customs officials for examination. 



Examining high-risk container shipments. 

CBP officials request that host government officials 
examine containers with high-risk shipments to detect 
WMD or other items with a nexus to terrorism. Examining 
a container involves using nonintrusive inspection equip- 
ment, radiation detection equipment, or both to scan the 
container's contents. Typically, the radiation detection 
equipment is used, then large scale nonintrusive inspec- 
tion equipment, to scan the container's contents. The 
results of the scan will influence whether or not CBP 
requests that the host government conduct a physical 
search, during which a container is opened and its 
contents are removed for review. 



Source: GAO and CBP. 



CBP official conducting targeting activities at the NTC. 




Container scanned with non-intrusive imaging x-ray 
equipment at a CSI port. 
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CBP Collaborated on 
the DHS Strategy to 
Enhance International 
Supply Chain 
Security, and Met 
Goals for CSI 
Expansion and 
Increased Container 
Examination 



CBP has undertaken strategic planning to guide efforts to secure the 
international supply chain and, more specifically, to manage the CSI 
program. CBP contributed to an international supply chain security 
strategy DHS recently issued that builds on DHS's existing strategic 
framework for maritime security. In 2006 CBP enhanced its strategic plan 
for CSI by including three key elements missing from the plan's previous 
iteration, and has achieved two performance goals by expanding CSI 
locations and increasing the percentage of total U.S.-bound containers that 
pass through CSI seaports. Concurrently, CBP reported an increase in the 
number of high-risk containers examined by host governments 
participating in CSI. 



Recently Issued 
International Supply Chain 
Security Strategy Builds on 
DHS's Existing Strategic 
Framework for Maritime 
Security 



When it published the Strategy to Enhance International Supply Chain 
Security in July 2007, DHS filled a gap that had existed between broad 
national strategies and program-specific plans in the federal government's 
strategic planning framework for maritime security. Over the last 5 years, 
DHS has made progress in developing a multilayered strategic framework 
for securing the maritime domain, including the international supply chain. 
This framework consists of high-level national strategies, such as the 
National Strategy for Maritime Security and the Maritime Commerce 
Security Plan, which describe the federal government's broad approach to 
maritime security. These plans are supplemented by a related hierarchy of 
documents that includes the DHS strategic plan, the CBP strategic plan, 
and the CSI program's own strategic plan. 

Prior to July 2007, the federal government's maritime security framework 
touched on many specific aspects of maritime trade and commerce, such 
as how the CSI program contributes to securing containers bound for U.S. 
seaports. However, it did not provide a detailed description of how 
federal, state, and local authorities were to collaborate on supply chain 
security specifically. In addition, Congress included a provision in the 
SAFE Port Act of 2006 requiring DHS to develop a strategic plan to 
enhance the security of the international supply chain. Moreover, the DHS 
fiscal year 2007 appropriation act withheld $5 million from DHS until a 
comprehensive strategic plan for port, cargo, and container security, 
which included specific elements, had been submitted to specified 
congressional committees. In response, CBP contributed to the Strategy 
to Enhance the International Supply Chain Security, which DHS 
developed and issued in July 2007. According to DHS, the supply chain 
security strategy is not meant to replace other strategic planning 
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documents, but seeks to harmonize the goals of the various plans and 
programs into a multilayered, unified approach that can be further 
developed by DHS components, including CBP. 

This new strategic planning document for supply chain security delineates 
the supply chain security roles, responsibilities, and authorities of federal, 
state, local, and private sector entities. The strategy seeks to build on the 
current multilayered strategic framework for maritime security by 
establishing an overarching framework for the secure flow of cargo 
through the supply chain — from point of origin to final destination. The 
strategy describes how CBP's portfolio of supply chain security 
initiatives — including CSI, C-TPAT, cargo screening using ATS, the 24-hour 
rule, and the use of nonintrusive inspection equipment to examine 
containers — addresses the various stages in the supply chain. In addition, 
the strategy provides details on how other organizations' programs or 
efforts — such as DOE's Megaports initiative, which places radiation 
detection equipment at foreign seaports — contribute to different aspects 
of supply chain security. Figure 4 describes the major components of the 
supply chain and the CBP initiatives that operate to secure them. 
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Figure 4: CBP Initiatives in the U.S. Supply Chain Security Strategy 
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Source: GAO. 



CBP Added Key Elements 
to the CSI Strategic Plan in 
Response to our 2005 
Recommendation 



At the program level, CBP has revised its CSI strategic plan, an important 
component of the DHS strategic framework described above, 
incorporating three critical elements that were absent from the plan's 
previous iteration. In our April 2005 report on CSI, we reported that the 
CSI strategic plan lacked three of the six key elements identified by the 
Government Performance and Results Act of 1993 for an agency strategic 
plan, including descriptions of 



1. how performance goals and measures are related to program 
objectives, 
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2. the external factors beyond the control of CBP that could affect 
the achievement of program objectives, and 

3. the evaluations that CBP conducts to monitor CSI. 6 

We noted that, given the importance of having an effective strategic plan 
for the program, we would continue to monitor CBP's progress in refining 
the plan. CBP has subsequently taken steps to address our concerns. In the 
most recent version of the plan, released in August 2006, CBP included 
information in three areas, as we had previously recommended. First, the 
CSI strategic plan links each performance measure to the strategic goal it 
supports. In addition, the plan describes how some performance measures 
were designed to act as proxies for program objectives that can be difficult 
to measure. Second, the CSI strategic plan also lists a variety of external 
factors that have the potential to influence CSI operations, including 
regional conflicts, organized crime, and changes in the political 
administration of a foreign government participating in CSI. Finally, the 
revised plan provides an explanation of the CSI team evaluation process, 
thus addressing the third issue identified in our April 2005 report. We 
discuss performance measure outcomes, other external factors, and CBP's 
evaluation process in greater detail later in this report. 



CBP Met Performance 
Goals to Expand Number 
of CSI Seaports and to 
Increase Proportion of 
Total U.S.-bound 
Containers Passing 
Through CSI Seaports 



The August 2006 CSI strategic plan set specific goals for expanding the 
number of seaports participating in CSI, and set targets for related 
increases in the percentage of total U.S.-bound containers that pass 
through CSI seaports. As of September 2007, CBP reported meeting its 
goals in both of these areas. Specifically, the plan called for CBP to expand 
CSI program operations from 40 to 50 seaports by the end of fiscal year 
2006, and to 58 seaports by the end of fiscal year 2007 (see appendix III for 
a complete list of participating seaports). Having reached its goal of 58 CSI 
seaports, CBP officials reported it currently does not have plans to add 
other CSI seaports, as the costs associated with expanding the program 
further would outweigh the potential benefits. In addition, the plan set a 
performance target that by 2010, 86 percent of all U.S.-bound container 
cargo was to pass through CSI seaports. 7 According to CBP, when U.S.- 



" Pub. L. No. 103-62, 107 Stat. 285. 

7 According to CBP, the remaining 14 percent of U.S.-bound containers that do not pass 
though CSI seaports are targeted by CBP officials at the NTCC to identify high-risk 
container shipments, and containers would be examined upon arrival at U.S. seaports if 
deemed necessary. 
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bound containers pass through CSI seaports there is an opportunity for 
high-risk cargo to be examined at the foreign seaport by the host 
governments participating in CSI, rather than upon arriving at a U.S. 
seaport. CBP reported that about 73 percent and about 80 percent of total 
U.S.-bound container cargo passed through CSI seaports in fiscal years 
2005 and 2006, respectively, and that it reached its 2010 goal early by 
reaching approximately 86 percent by the end of fiscal year 2007. Figure 5 
shows that as the number of operational CSI seaports expanded from 2002 
to 2007, the proportion of total U.S.-bound container cargo passing 
through CSI seaports also continued to increase. 8 



The increase in the percentage of total U.S.-bound containers passing through CSI 
seaports is not proportional to the number of CSI seaports in the program because the 
volume of U.S.-bound containers varies from seaport to seaport (and year to year). 
Appendix III provides details on when specific foreign seaports began conducting CSI 
operations. 
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Figure 5: Number of Operational CSI Seaports and Percentage of Total U.S-bound 
Containers Passing Through CSI Seaports, 2002-2007 
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Source: GAO presentation of CBP data. 

In implementing the CSI program and reaching its goal of 58 operational 
CSI seaports, CBP selected foreign seaports to participate in the program 
in three phases. CBP officials reported using the following general 
selection criteria for each phase as follows: 9 

• Most of the 23 phase I seaports were selected because they shipped the 
highest volume of U.S.-bound container cargo. 10 



According to CBP officials, due to logistics such as the time necessary for negotiations 
with host governments and staffing CSI teams in foreign countries, CSI seaports selected in 
phases one and two sometimes did not begin operations until later selection phases were 
underway. 

10 According to CBP, phase I included three Canadian seaports at which there was already a 
customs relationship with the United States. 
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• The 19 phase II seaports were selected based on factors such as cargo 
volume, strategic threat factors and the foreign government's level of 
interest in CSI. 

• The 16 phase III seaports were selected using the phase II criteria as 
well as diplomatic or political considerations, such as the requests of 
foreign governments already participating in CSI. 

As CBP expanded the number of CSI seaports and increased the 
proportion of total U.S.-bound container cargo passing through CSI 
seaports, the agency also achieved increases in security activities that 
occur at CSI seaports — targeting (CBP screens container cargo with ATS 
to produce risk scores and conducts additional review or research to 
ascertain risk levels) and examining high-risk container cargo (host 
government officials examine high-risk containers by scanning with 
nonintrusive inspection equipment or by physically searching the 
container). As of September 2007 CBP reported fully targeting 100 percent 
of all U.S.-bound container cargo to identify high-risk cargo as required by 
the SAFE Port Act. 11 In addition, foreign governments participating in CSI 
have examined an increasing amount of high-risk container cargo as a 
growing proportion of total U.S.-bound containers pass through CSI 
seaports. In keeping with the CSI program's risk-based approach, CBP 
currently does not request that the host governments examine all U.S.- 
bound containers passing through the CSI seaports, just those that CBP 
officers have determined to be high-risk. In fiscal year 2006, the number of 
high-risk containers examined by host government officials at CSI 
seaports increased by 77 percent from the previous year to almost 71,000 
containers. In fiscal year 2007, examinations continued to increase, 
reaching almost 137,000 containers. Moreover, in fiscal year 2007 CBP 
reported that host government officials examined approximately 96 
percent of the container cargo referred for examination. CBP reported that 
about 4 percent of the referrals did not lead to examinations (about 5,600 
requests) because (1) logistical difficulties arose, such as the container 
had already been loaded on the shipping vessel (about 5,200 requests), or 
(2) the host government denied the request (fewer than 400 requests). 



CBP reported that NTCC targeters assist the CSI teams at high-volume seaports to help 
ensure all containers that pass through CSI seaports are targeted. 
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To Strengthen CSI 
Operations, CBP Has 
Taken Steps to 
Address Human 
Capital Challenges 
and Enhance Host 
Government 
Relations, but 
Operational 
Challenges Remain 



CBP has made various operational improvements to CSI, though 
challenges remain. First, CBP has revised its human capital plan and 
added permanent staff at CSI seaports, though it reports difficulties in 
hiring and deploying qualified staff. Second, CBP's relations with CSI host 
governments we spoke to that conduct cargo examinations have improved 
over time, though access to key examination-related information and 
processes is limited by host governments at some CSI seaports. And 
finally, CBP's ability to conduct CSI program activities involves logistical 
challenges that are inherent to many seaport environments, such as those 
that are densely packed with equipment and personnel. 



CBP Has Increased 
Permanent Staffing Levels 
at CSI Seaports, but Has 
Yet to Determine Optimum 
Distribution of Staff to 
Ensure All Critical 
Operations Are Performed 



The ability of the CSI program to operate in accordance with its mission 
and objectives depends, in part, on the success of its human capital 
strategy — and CBP's ability to manage and deploy staff in a way that 
ensures that critical security functions are performed. Our April 2005 
report on CSI noted that although CBP's goal is to target all U.S.-bound 
cargo shipped in containers at CSI seaports before they depart for the 
United States, the agency had not been able to place enough officers at 
some CSI seaports to do so. Specifically, CBP had developed a CSI staffing 
allocation model to determine the staff needed to target container cargo. 
However, at some CSI seaports CBP had been unable to staff the CSI 
teams at the levels called for in the CSI staffing model. 12 We noted that 
CBP's staffing model had not, at the time, considered whether some of the 
targeting functions could be performed in the United States. We 
recommended that CBP revise its staffing model to consider what 
functions need to be performed at CSI seaports and what functions can be 
performed in the United States, optimum levels of staff at CSI ports, and 
the cost of locating CBP targeters overseas at CSI seaports instead of the 
United States. 



CBP has subsequently taken several steps to increase the number of CSI 
officers and to implement our 2005 recommendations. For example, in 



CBP's staffing model calculates the number of officers required at a CSI seaport as equal 
to the annual volume of containers shipped to the United States divided by 64,350 (half the 
number of containers a CSI team member should be able to target in a given year) plus or 
minus the level of risk associated with that CSI seaport. 
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response to our concerns about staffing imbalances across seaports and 
shortages at the highest-volume seaports, CBP has increased staffing 
levels, bringing them closer to those called for in its staffing model — 
resulting in a parallel increase in the volume of container cargo that is 
targeted. Also, CBP has added 15 staff to CSI targeting duty at the NTCC 
since 2005, composed of temporary and permanent officers. In addition, in 
fiscal year 2007 CBP deployed an additional 125 permanent and 68 
temporary officers to CSI seaports. Considering the officers at both CSI 
seaports and the NTCC, as of November 2007, CBP had deployed 209 CSI 
officers, which exceeds the 203 called for in the CSI staffing model. As a 
result of these efforts, CBP officials told us that they had increased their 
targeting of U.S.-bound container cargo from 65 percent in April 2005 to 
100 percent in September 2007. 

The agency also developed cost estimates for placing a mix of permanent 
and temporary staff at CSI seaports (with permanent staff costing about 
$330,000 per year and temporary staff about $275,000 per year) in response 
to our recommendation. CBP reported that the advantages of placing 
officers at CSI seaports on a permanent rather than a temporary basis 
include greater opportunities for enhanced communication and 
coordination with host governments, and less disruption due to fewer 
rotations into and out of the country. At one CSI port that we visited, host 
government customs officials told us that the presence of permanent staff 
facilitated increased information sharing, which over time could lead to a 
decrease in unnecessary examinations. 

Despite the progress it has made, CBP continues to face staffing 
challenges. CBP officials told us, for example, they continue to face 
challenges in obtaining sufficient numbers of qualified officers to be 
permanently deployed at CSI seaports. For example, CBP officials 
reported that only 9 qualified applicants applied for 40 permanent 
positions at CSI seaports. Officials told us that CSI must compete for staff 
with targeting or seaport experience with other CBP programs or 
positions, such as C-TPAT or other programs that operate at the NTCC. To 
fill open positions at CSI seaports, CBP officials reported that in some 
instances officers have been deployed who have not received all of the 
required training. In addition, CBP evaluation data we reviewed showed 
examples of CBP officers at CSI seaports lacking key skills, such as the 
ability to target proficiently or communicate in the local language. 

In addition, CBP has taken action to enhance its human capital planning 
process for CSI, but has not yet included important factors in its staffing 
allocation model. As we reported in 2005, one of the features of the CSI 
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staffing model that may contribute to staffing imbalances was its reliance 
on placing officers overseas at CSI seaports. It did not consider what 
functions could be done in the United States. In May 2006, in response to 
our recommendations, CBP issued a human capital plan that did not 
specify that CSI targeting positions be located at CSI seaports, thus 
recognizing that officers could support CSI seaports from the NTCC in the 
United States. CBP officers assigned to the NTCC perform many of the 
same roles as officers at CSI seaports, including reviewing bills of lading. 13 
CBP officers at the NTCC review bills of lading for high-volume seaports 
where the placement of the number of CSI officers required to review all 
bills of lading is unfeasible. 14 In addition, according to CBP officials, CBP 
officers at the NTCC review bills of lading for U.S.-bound cargo from CSI 
seaports where no CBP officers are stationed. Though CBP's 2006 human 
capital plan generally recognizes that some CSI functions can be 
performed at either a CSI seaport or at the NTCC, the staffing allocation 
model used to calculate the number of targeters necessary to review bills 
of lading for each CSI port does not include factors that specify where 
these positions should be located. 

In addition, CBP's staffing allocation model does not take into account 
activities other than targeting — such as witnessing host government 
examinations — that CSI officers perform at CSI seaports. According to 
CBP, the agency stations as many of the total officers needed as possible 
at the CSI seaports, but if the number of officers needed is higher than the 
number of officers allowed by the host government or available to be 
stationed in the seaport, then the remainder of the officers target from the 
NTCC. 15 However, we found that CBP has still not systematically 
determined the optimal number of officers that need to be physically 



Bills of lading are documents issued by carriers describing cargo in a shipment, details of 
the intended voyage, and the conditions of transportation. 

14 Host nations may limit the number of CSI personnel to less than optimum per the CSI 
staffing model. According to CBP officials, since 2005, CBP has reported it is unable to 
staff the CSI teams at the levels called for in the CSI staffing model because of diplomatic 
and practical considerations. For example, the host government or the U.S. Department of 
State can restrict the size of the CSI teams located at foreign seaports. 

15 Permanent recruitment for all overseas positions requires DHS and State Department 
approval through the National Security Decision Directive 38 (NSDD38) process. The 
NSDD38 provides Chiefs of Mission the authority to determine the size, composition, and 
mandate of personnel operating under their authority. The NSDD38 process is required 
whenever a requesting agency is establishing or abolishing an activity, and changing the 
size, composition, or mandate of full-time permanent direct-hire positions. 
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located on-site at CSI seaports to carry out duties that require an overseas 
presence (such as coordinating with host government officials or 
witnessing the examinations they conduct) as opposed to other duties that 
could be performed off-site in the United States (such as reviewing bills of 
lading and databases). Also, CBP's revised CSI human capital plan does 
not include costs related to placing temporary staff at the NTCC and thus 
does not have the data needed to conduct a cost-benefit analysis for 
determining the optimal location for its CSI officers. 16 As we noted in our 
2002 report on a staffing framework for use at U.S. embassies, federal 
agencies should consider factors such as cost and physical security of 
foreign operations and consider options such as relocating staff to the 
United States, as part of their framework for determining the right number 
of staff to be placed overseas. 17 Determining optimal staffing levels is 
particularly important in light of ongoing challenges CBP reports facing to 
identify sufficient numbers of qualified individuals to hire for the program, 
and in light of the program's recent expansion to additional seaports 
around the world. 

While CBP has taken steps to implement the recommendations from our 
April 2005 report, further action is needed regarding the staffing allocation 
model. Specifically, as we recommended in 2005, the model should be 
revised to consider (1) what functions need to be performed at CSI 
seaports and what functions can be performed in the United States, (2) the 
optimum levels of staff needed at CSI seaports to maximize the benefits of 
targeting and examination activities in conjunction with host nation 
customs officials, and (3) the cost of locating targeting positions overseas 
at CSI seaports instead of in the United States. 



Level of Collaboration 
between U.S. and Host 
Customs Officials Has 
Improved, but Challenges 
Remain at Some CSI 
Seaports 



The CSI human capital plan states that CBP has relied on CBP officers on temporary duty 
at the NTCC for CSI duties. 

17 GAO, Overseas Presence: Framework for Assessing Embassy Staff Levels Can Support 
Rightsizing Initiatives, GAO-02-780 (Washington, D.C.: July 2002). 



CSFs strategic plan emphasizes the importance of CBP's continued efforts 
to foster partnerships with foreign customs officials at CSI seaports to 
improve CSI operations. Specifically, according to CBP headquarters 
officials, when CSI teams stationed at foreign seaports develop strong 
interpersonal relations with foreign government officials, it leads to 
increased trust and information sharing and thus improved targeting and 
examination of high-risk cargo. While the extent of cooperation across all 
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of the 58 CSI seaports now operating is difficult to quantify, our 
observations at 6 CSI seaports and our review of select CSI team 
evaluations provide examples of how collaboration can benefit the CSI 
program, and conversely, how the lack thereof can hinder progress. 18 

At all 6 CSI seaports we visited, CBP officers or host government officials 
told us that the relationship between the CSI team and the host 
government has been positive or has improved over time. CBP and host 
government officials we spoke with at all of the seaports we visited 
reported that establishing trust and collegiality has led to increased 
information sharing, resulting in more effective targeting and examination 
of high-risk container cargo. For example, CBP officers noted instances in 
which host customs officials would occasionally notify them of container 
cargo they thought could be high-risk, so that CBP could take a closer look 
at the information available in ATS related to the container cargo. In 
addition, a few CBP officers or host government officials stated that the 
presence of CSI teams at foreign seaports has in many instances helped to 
prevent unnecessary examinations because information provided by host 
government customs officials has led to lower risk profiles for certain 
container cargo. 

Moreover, CBP officials reported that strengthened relationships with host 
government officials and the trade community have led host governments 
to bolster their customs and port security practices. CBP officials we 
spoke to emphasized that, like the United States, most foreign customs 
administrations have traditionally focused on revenue collection and the 
seizure of contraband, rather than security concerns. During our visits to 
CSI seaports, the CBP and host government officials we spoke with 
reported several examples of how the presence of CSI teams at seaports 
has helped to expand the focus of the efforts of these foreign customs 
administrations and the trade community to include enhanced security 
practices. For example, one country developed databases with trade 
information to achieve its customs goals and to assist CSI after seeing how 
gathering historical data benefited CBP. Furthermore, at a couple of the 
CSI seaports we visited, the CSI team or host government officials 
arranged outreach meetings with the trade community to raise companies' 
awareness of security practices and the benefits of providing correct and 
complete data about their cargo. 



The methods we used to evaluate CBP information in CSI team evaluations and to 
observe operations at CSI seaports we visited are described in appendix I. 
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During our visits to CSI seaports and our review of data CBP collected 
during its evaluations of CSI teams, we also identified instances where 
cooperation between CSI teams and their counterparts in the host 
government could be improved — though, as CBP officials noted, some of 
the factors involved are beyond CBP's ability to control directly. For 
example, in some locations, CBP officials reported that a country may 
have laws that hinder the collaboration of host government officials with 
CSI teams. We identified the following issues during our observations at 6 
CSI seaports as well as from our review of CBP data collected in fiscal 
year 2007 at an additional 12 CSI seaports (for a total of 18 CSI seaports): 

• At 9 CSI seaports, the CSI teams there reported that they only 
interacted infrequently with their host government counterparts or the 
host government officials did not readily share information that would 
benefit CSI, such as knowledge about potentially suspicious container 
cargo. In one instance the lack of interaction was attributed to the host 
government's competing priorities. 

• At 6 CSI seaports, host governments restricted CSI teams from viewing 
nonintrusive inspection equipment examinations conducted by host 
customs authorities or the resulting images of the container's contents, 
which is one of the key purposes for staffing CBP officers at CSI 
seaports. 

• At 4 CSI seaports, host governments prohibited the use of hand-held 
radiation detection devices by CBP officials, which is considered by 
CBP to be an important way to identify a potential anomaly in a high- 
risk container. According to CBP officials, a few of the countries 
prohibit the equipment due to safety and health concerns about the use 
of the equipment. 

• At 3 of the CSI seaports, host customs officials lacked access to 
technical equipment, such as computers or nonintrusive inspection 
equipment that worked properly, which CBP believes could limit their 
ability to share customs-related information with CSI team members or 
efficiently conduct examinations. According to CBP officials, 
sometimes host governments lack resources to meet these 
technological needs. 

• At 6 CSI seaports in 2 countries, CBP officers at the seaport reported 
that host customs administrations did not provide a sufficient number 
of staff to assist CSI teams or the host government officials were often 
unavailable, which, according to CSI teams, can sometimes lead to 
delays in examining high-risk containers. 
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• At 3 CSI seaports, there was evidence of challenges to effective 

communication, such as some CSI teams having limited proficiency in 
the local language. 19 

These examples are not intended to represent the CSI program as a whole, 
but are included to illustrate the types of challenges that CSI teams at the 
seaports and CBP program managers face. CBP officials responsible for 
managing the CSI program have reported that overall there has been a 
high level of cooperation at CSI seaports, though they acknowledged that 
the degree of involvement and participation that CBP officers have with 
foreign customs officials during the examination of high-risk cargo varies 
by country. It is also important to note that while CBP negotiates a 
written, nonbinding arrangement stating expectations for inclusion in the 
CSI program with the participating foreign governments, the agency 
cannot compel foreign governments to offer information for the purposes 
of CSI or to examine high-risk containers. Later in this report, we describe 
the processes CBP has in place to address difficulties that may be 
identified at the CSI seaports as part of its program oversight and 
monitoring efforts. 



Seaport Environment and 
Logistics Present 
Challenges to CSI 
Operations 



Another factor that can affect CBP's ability to conduct CSI program 
operations involves logistical challenges that are inherent to many seaport 
environments. For example, as illustrated in figure 6, foreign government 
officials we spoke with at CSI seaports reported that many seaports are 
densely packed with equipment and personnel, which can make it difficult 
for host government customs officials to examine container cargo. 



According to CBP, all permanent CSI staff deployed at CSI seaports receive some foreign 
language training. However, a minimum proficiency in the language is not required. 
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Figure 6: View of the Physical Layout of a Congested CSI Seaport 




Source: GAO. 



According to CBP, open space to place scanning equipment or to conduct 
physical searches of containers can be scarce at some CSI seaports. For 
example, in two of the CSI locations we visited, scanning equipment and 
examination sites were placed several miles from where container cargo is 
unloaded, loaded, or stored. According to the CBP officials we spoke with, 
this adds to the costs and time required for examination and may result in 
logistical difficulties in having high-risk U.S.-bound containers examined 
before being loaded onto the shipping vessel. In addition, at one port we 
visited, the host government limited the number of containers it would 
examine, in part to limit the cost of examination and the amount of delay 
caused by moving these containers, according to the CSI team we spoke 
with. CBP officials reported that despite this limit to examine no more 
than 250 containers (out of the over 115,000 container cargo shipments to 
the United States from this seaport in fiscal year 2007), the country has not 
denied many examination requests — only two in fiscal year 2007. 
However, this ceiling was not based on risk factors, and an increase in 
denied requests could lead to additional containers with high-risk cargo 
departing for the United States without being examined. 
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Finally, CBP officials stated that containers at seaports are generally 
stored in a container yard before they are loaded onto the shipping vessel. 
These container yards may be very large, and containers in these yards are 
often stacked to minimize the time required to load container vessels. As 
shown in figure 7, containers on a vessel may be stacked several layers 
deep. Accordingly, CBP and host government officials we spoke to at a 
few CSI seaports reported it can sometimes be challenging to access a 
container for examination. CBP officials noted that any examinations 
requested but not conducted in the CSI seaport would occur at a U.S. 
seaport upon arrival. 



Figure 7: Stacked Containers on a Shipping Vessel at a CSI Seaport 




Source: GAO. 
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CBP Has Enhanced 
Its CSI Evaluations at 
CSI Seaports and 
Performance 
Measures but Still 
Does Not Capture 
Critical Information 
about Host 
Government 
Examination Systems 



CBP has enhanced how it collects CSI data by strengthening its approach 
to conducting periodic evaluations of CSI officers at CSI seaports through 
on-site evaluations of performance. However, weaknesses remain in how 
CBP conducts evaluations, the information collected regarding host 
government examination systems, and performance measurement of the 
program as a whole. 20 For example, CBP does not systematically collect 
information on the equipment, people, and processes that are part of the 
host government's overall examination system. Also, while CBP has 
refined and updated its performance measures, we identified remaining 
limitations, such as the omission of measures for all core program 
elements and several performance targets. 



CBP Significantly 
Improved Its CSI 
Evaluations to Assess 
Program Operations at CSI 
Seaports, but Weaknesses 
Remain 



CBP conducts evaluations at CSI seaports to determine the effectiveness 
of the program. Specifically, CBP uses these on-site evaluations to assess 
CSI team operations and capabilities, such as how well CSI team members 
use ATS to determine the risk levels associated with U.S.-bound 
containers passing through CSI seaports. CBP's CSI strategic plan states 
that these periodic reviews are intended both to ensure that deployed CSI 
teams are adhering to standard operating procedures as well as to evaluate 
the relationships between the teams and the host customs administrations. 
In fiscal years 2006 and 2007, CBP reported conducting 42 and 45 
evaluations, respectively. Since the program's inception in 2002, the 
agency reported conducting a total of 202 evaluations. 

In November 2006, CBP significantly changed the way it conducts CSI 
team evaluations. Prior to that time, CBP officials reported that its 
evaluators relied on self-reported information from CSI team members on 
how proficiently they performed CSI program activities. CBP's current 
approach to conducting CSI team evaluations seeks to provide a more 
thorough review of CSI team performance. According to CBP officials, the 



We use the term "examination system" to refer to the overall equipment, people, and 
processes used by any country to assess goods leaving or entering their seaports. In the 
context of CSI, U.S. and host government officials share the role of assessing goods leaving 
the seaports of countries participating in CSI. CSI teams at CSI seaports are responsible for 
targeting high-risk containers and other tasks, whereas host government customs officials 
examine containers by scanning a container with nonintrusive inspection equipment, a 
physical search of the container's contents, or both. 
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agency now requires the CSI team members under review to demonstrate 
their targeting competence to an evaluator, such as by physically showing 
the evaluator how they review information about container cargo to 
determine its risk level. To better assess the deployed CSI team's 
performance, CBP augmented its evaluation teams with officers who have 
expertise in areas such as targeting and intelligence gathering. 

Also, CBP has developed a new software tool that enables evaluators to 
record evaluation data electronically, using laptop computers to conduct 
the on-site evaluations. This tool, CSI Team Evaluation (CSITE), consists 
of a series of yes or no questions that cover the various areas of CSI team 
performance, including whether all of the container cargo that the CSI 
team designated high-risk were examined and whether these actions were 
properly documented. The CSITE tool also provides guidance on each 
question and prompts evaluators as they conduct their review by, for 
example, directing them to ensure that the CSI team is using the correct 
settings in ATS. In addition, employing CSITE, CBP reported it can now 
aggregate the results of some or all of its evaluations, a capability it 
previously lacked, and can conduct statistical analyses of the results of the 
evaluations. The agency can determine, for example, what percentage of 
CSI team members successfully demonstrated proficiency in targeting 
high-risk containers. According to CBP officials, CSITE will eventually 
allow the agency to make comparisons of CSI performance across 
seaports. Moreover, CBP now retains the information it collects at CSI 
seaports and the resultant evaluation reports in a more systematic fashion. 
CBP officials acknowledged that the agency did not always store this data 
effectively prior to the implementation of the new evaluation system and 
could not provide us with documentation of all of the evaluations it had 
conducted since the program's inception. 

While these efforts should help to strengthen the CSI team evaluation 
process, CBP is still not consistently collecting all available data to aid in 
its analysis of CSI team performance, and we identified instances in which 
the agency did not reconcile contradictory information it had collected. 
Based on our review of CBP's documentation associated with 34 
evaluations to assess the information the agency collected and its methods 
for doing so, we found that evaluators do not always answer all of the 



Page 34 



GAO-08-187 Supply Chain Security 



questions contained in CSITE. 21 For example, the software tool instructs 
the CBP evaluation team to collect information on whether 
recommendations made in prior evaluations have been implemented. This 
information could allow CBP to determine whether past problems have 
been addressed, but it is not always provided by the evaluation team. We 
also identified discrepancies between (a) the CSITE checklist of questions 
that the evaluation team completes during the onsite evaluation, and (b) 
the resulting evaluation report produced by CBP headquarters officials for 
2 of the 14 locations for which we had both documents to compare. At one 
seaport, for example, the CBP evaluation team indicated in the CSITE 
checklist that the CSI team did not have all of the data systems it needed 
to effectively target outbound shipments, whereas the evaluation report 
stated the team had access to all of the appropriate targeting tools and 
databases. With more complete information, collected in a consistent 
manner, CBP may be better able to determine how well CSI teams are 
performing, what corrective actions may be needed to improve the 
program, or whether the CSI program is achieving its security goals. 



CBP Lacks a Process for 
Systematically Gathering 
Information on Host 
Government Examination 
Systems, Which Include 
Equipment, People, and 
Processes 



Host Government 
Examination Systems- 
Equipment 



In April 2005, we recommended that CBP establish minimum technical 
criteria required for the capabilities of nonintrusive inspection equipment 
at CSI seaports, while considering sovereignty issues with participating 
countries. CBP agreed to evaluate the feasibility of establishing such 
criteria. In 2006, section 205(e) of the SAFE Port Act required DHS to 
establish minimum technical capability criteria for the use of nonintrusive 
inspection equipment and nuclear and radiological detection systems in 
conjunction with CSI, but noted that these criteria should not be designed 



This sample was composed of all of the evaluations that had been conducted using the 
CSITE tool at the time of our review, the evaluations that directly preceded them 
chronologically (where available), and one evaluation from each additional seaport for 
which we had documentation. See appendix I for more details about our selection 
methodology. 
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to conflict with the sovereignty of host countries. In 2007, the 9/11 Act also 
required the Secretary of DHS to develop technological standards for 
scanning systems that will be used to conduct 100 percent scanning at 
foreign seaports in the future and to ensure that these and other actions 
implementing the act's 100 percent scanning provisions do not violate 
international trade obligations and are consistent with the World Customs 
Organization framework or other international obligations of the United 
States. 22 CSI host governments, which are responsible for conducting 
examinations of container cargo, purchase and operate nonintrusive 
inspection equipment, though as of November 2007, 13 CSI seaports use 
equipment on loan from the United States. The capabilities of this 
inspection equipment vary by manufacturer and model. The equipment 
may differ, for example, in its ability to penetrate steel shielding in order 
to generate an image of container contents, or may scan containers at 
different rates. Appendix IV describes the capabilities of this equipment in 
greater detail. As of November 2007, CBP had not yet implemented our 
prior recommendation or taken actions to meet the SAFE Port and 9/11 
Acts requirements for setting minimum technical criteria. CBP officials 
stated that the reason for this is that they do not consider the agency to be 
a standard-setting organization. While CBP refers host governments to the 
World Customs Organization' SAFE Framework regarding the 
procurement of inspection equipment, this document does not include 
specific technical criteria or standards. Moreover, they added that it is 
important to acknowledge the inherent challenges involved in efforts to 
ascertain the capabilities of nonintrusive inspection equipment that is 
owned and operated by CSI host governments. 

In May 2005, however, CBP put forth minimum technical criteria to 
evaluate the quality and performance of nonintrusive imaging inspection 
equipment being considered for use at U.S. seaports. 23 These domestic 
standards set baseline performance requirements for penetration, contrast 



This provision appears to refer to the Framework of Standards to Secure and Facilitate 
Global Trade, commonly referred to as the SAFE Framework, which was adopted by the 
member countries of the World Customs Organization, including the United States, in June 
2005. As of September 11, 2007, 148 member countries had signed letters of intent to 
implement the SAFE Framework. 

23 There are two types of nonintrusive inspection equipment currently used at CSI seaports: 
(1) radiation detection equipment and (2) imaging inspection equipment, which may use X- 
rays or gamma rays. Radiation detection equipment, such as a radiation portal monitor, 
detects radioactive emissions that may originate from a container, indicating the presence 
of radiological material. 



Page 36 



GAO-08-187 Supply Chain Security 



sensitivity, throughput, image quality, and scan size. To determine whether 
certain types of nonintrusive inspection equipment were acceptable for 
use at domestic seaports — and could meet the criteria that had been set — 
CBP conducted tests comparing the capabilities of nonintrusive imaging 
inspection equipment provided by seven manufacturers with its technical 
operating standards. On the basis of the test results, CBP recommended 
the inspection equipment from five of the seven manufacturers for use at 
domestic seaports, while equipment from two manufacturers was not 
recommended. CBP officials stated that there are no plans to 
systematically compare the capabilities of inspection equipment at CSI 
seaports against these criteria for domestic equipment due to sovereignty 
concerns. 

CBP collects limited information on certain characteristics of the 
inspection equipment installed at CSI seaports, such as manufacturer; 
however, information related to capabilities and performance is not 
generally obtained. Officials in CBP's Office of Technology stated that they 
have information on the capabilities of equipment that the United States 
loans to other countries for 16 CSI seaports, and that only this equipment 
can be assured of meeting the CBP domestic requirements. However, 
these CBP officials said that they had neither determined which other CSI 
seaports use the inspection equipment that was assessed as part of CBP's 
test and recommended for use at domestic seaports, nor systematically 
determined the specific capabilities of the equipment used at those CSI 
seaports. Host government officials in the countries we visited stated that 
they followed their country's acquisition procedures, which included 
reviewing equipment capabilities and performance, among other things, 
for the purchase of nonintrusive imaging inspection equipment. However, 
CBP does not have documentation on the testing used by the host 
countries or the manufacturers to determine the basis for the equipment's 
stated performance or whether this stated performance is less than, meets, 
or exceeds the criteria CBP established for equipment used at domestic 
seaports. 

According to CBP officials, the capabilities of nonintrusive inspection 
equipment are vetted during an assessment phase of the CSI program, 
when CBP is determining whether a seaport is prepared to operate within 
CSI. While, as part of the assessment phase, CSI officials stated that they 
collect descriptive technical information about the type of nonintrusive 
inspection equipment to be used at seaports, we did not find — in our 
review of CBP's checklist used to guide its assessment teams as they 
examine prospective CSI seaports — questions covering inspection 
equipment other than general direction to ascertain whether some type of 
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this equipment was in place. Also, through our review of CBP's 
assessments of 10 CSI seaports — through which approximately 55 percent 
of all U.S.-bound containers passed in fiscal year 2007 — we did not find 
any assessments that described the performance capabilities of the 
equipment or judgments about the proficiency of host government officials 
in operating these systems. CBP officials stated that the agency has never 
prohibited a seaport from participating in CSI on the basis of its inspection 
equipment, and CBP documents show that participation in the program 
requires only that some type of nonintrusive inspection equipment be 
available at or near the potential CSI port. 

Host Government The SAFE Port Act also directed DHS to (1) establish standard operating 

Examination Systems— People procedures for the use of nonintrusive inspection equipment at CSI 
and Processes seaports and (2) require CSI seaports to operate the equipment in 

accordance with the criteria and operating procedures established by 
DHS. 24 Also, the 9/11 Act required DHS to develop operational standards 
for scanning systems that will be used to conduct 100 percent scanning at 
foreign seaports in the future. CBP officials stated that they recognize that 
the capabilities of nonintrusive inspection equipment are only one element 
for determining the effectiveness of examinations that take place at CSI 
seaports. It is better, in their view, to make assessments of the whole 
examination system, which includes nonintrusive inspection equipment, 
personnel, and processes. However, CBP acknowledged it does not 
systematically collect information on host governments' use of 
examination systems and has not developed general guidelines or criteria 
that could provide CBP with the means to determine the quality of 
examinations of high-risk container cargo bound for the United States. 
CBP officials stated that they rely on CSI teams to notify headquarters if 
they have concerns about the host government customs or examination 
practices. Specifically, each CSI team leader is to meet weekly — usually 
via teleconference — with a CSI manager located at CBP headquarters to 
discuss ongoing CSI operations. However, CBP officials acknowledged 
that equipment, capabilities, and examinations practices of host 
government customs personnel are not routinely discussed. 

CBP officials also reported that CSI team members witness most 
examinations of high-risk U.S.-bound containers, and their presence at the 



The act stated that the technical criteria and operating procedures should not be 
designed to conflict with the sovereignty of host countries, but it did not address host 
government sovereignty related to requirements for CSI seaports to operate the equipment 
in accordance with the criteria and procedures. 
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examinations would allow them to make judgments about aspects of the 
host government's examination system. However some host governments 
specifically prohibit CSI team members from witnessing examinations. 
Also we found that CBP officials did not routinely observe inspections at 
one CSI seaport we visited, and were not always able to be present for 
inspections at two other CSI seaports because those inspections were 
scheduled and conducted when CBP officials were not available. 

CBP officials told us that their CSI team evaluations are also a means of 
capturing some information on various aspects of the host government's 
examination system. In order to participate in CSI, CBP requires that, 
among other things, host governments have customs staff capable of 
examining cargo originating in or transiting through its country and 
maintain a program to prevent breaches in employee integrity. However, 
the 15 CSI team evaluations we reviewed, which CBP had conducted since 
the agency revised its evaluation process in November 2006, showed 
limited coverage of whether host government customs personnel have 
been trained to use nonintrusive inspection equipment or are using it 
properly, the sufficiency of host staffing levels, and host government 
efforts to ensure the integrity of their customs administration. Specifically, 
6 of the 15 CSI team evaluations discussed whether equipment was used 
properly, 1 discussed host staffing levels, and none discussed host 
integrity programs. 



CBP's Lack of Information on 
Host Government 
Examination Systems 
Potentially Limits Assurance 
That Examinations ofHigh- 
Risk Container Cargo are 
Effective 



CBP's lack of a systematic way to collect information on host 
governments' examination systems — including their equipment, people, 
and processes — potentially limits CBP's ability to ensure that 
examinations of high-risk container cargo at CSI seaports can detect and 
identify WMD. Without information on host governments' examination 
systems, CBP management may not be able to determine the reliability of 
the host government's inspections of high-risk U.S.-bound container cargo. 
This is of particular concern since, according to CBP officials, most high- 
risk cargo that has already been examined at a CSI seaport, is generally 
not reexamined once it arrives at a U.S. seaport. 25 CBP officials stated that 
if problems are found in the examination process at a CSI seaport, then 



° According to CBP, containers are generally not reexamined in the United States unless 
new information is provided about risks or threats associated with the cargo, such as 
information related to its point of origin or some similar factor. 
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high-risk container cargo would be reexamined upon arrival in the United 
States. 

As already noted, CBP must respect participating countries' sovereignty. 
CBP cannot require that a country use specific equipment. However, if a 
high-risk container was examined using an examination system found by 
CBP to be less capable than established criteria, the agency could require 
that the container be reexamined upon arrival at a U.S. seaport. CBP 
officials stated that they believe that in general the equipment used by 
participating governments meets or exceeds the capabilities of the 
nonintrusive inspection equipment used at U.S. seaports. However, 
because CBP has not set minimum technical criteria for nonintrusive 
inspection equipment at CSI seaports, and the agency does not 
systematically review the operations of the host government examination 
systems at CSI seaports, CBP potentially has limited assurance that their 
inspection equipment is capable of detecting and identifying potential 
WMDs. In light of the new 9/11 Act requirement that 100 percent of U.S.- 
bound container cargo be scanned in the future with nonintrusive 
inspection equipment at foreign seaports before leaving for the United 
States, it is important that CBP have processes in place to gather the 
information necessary to ensure that cargo container examinations — and 
the equipment used as part of the examination process — are reliable, 
regardless of the point of origin. 



CBP Made Efforts to 
Refine CSI Performance 
Measures, but Did Not 
Fully Address our Previous 
Recommendation 



While CBP has taken steps to strengthen performance measures for the 
CSI program, we identified areas that did not fully address our April 2005 
recommendation to develop outcome-based performance measures or 
proxy measures of program functions — if program outcomes could not be 
captured — and performance targets to track the program's progress in 
meeting its objectives. Whereas CBP's CSI team evaluations and program 
monitoring activities help to evaluate CSI operations at the seaport level, 
CBP uses performance measures to gauge the effectiveness of the overall 
program in meeting its broader strategic objectives for CSI across 
seaports. By definition, performance measures are a particular value or 
characteristic used to quantify a program's outputs — which describe the 
products and services delivered over a period of time — or outcomes — 
which describe the intended result of carrying out the program. A 
performance target is a quantifiable characteristic that establishes a goal 
for each measure; agencies can determine the program's progress, in part, 
by comparing the program's measures against the targets. For example, 
the target of one of CBP's performance measures — the "number of 
operational CSI seaports" — was to have 58 CSI seaports operating in fiscal 
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year 2007, which the agency achieved as described previously in this 
report. The Government Performance and Results Act of 1993 
incorporated performance measurement as one of its most important 
features, and the establishment and review of performance measures are a 
key element of the standards for internal control within the federal 
government. 26 As discussed in the Government Performance and Results 
Act of 1993 and as we reported in 1996, measuring performance allows 
organizations to track progress being made toward specific goals and 
provides managers crucial information upon which to base their 
organizational and management decisions. 27 In addition, leading 
organizations recognize that performance measures can create powerful 
incentives to influence organizational and individual behavior. 

In the past 2 years, CBP has made efforts to refine and modify its 
performance measures as the CSI program has matured. Since 2005, for 
example, CBP has eliminated five performance measures that it had used 
to track the implementation of seaports participating in CSI, measures that 
CBP determined were no longer needed because CSI operations were 
under way at the majority of planned CSI seaports. Also, in our April 2005 
review of CSI, we identified a CSI performance measure that was 
calculated inappropriately, and in response, CBP modified how the 
measure was calculated to address our concerns. Specifically, for the CSI 
measure that tracks the number of container examinations waived 
because they are determined to be unnecessary, CBP began excluding 
inappropriate data that made the results of the performance measure 
misleading. 28 This was an important modification because, as we reported 
in November 2002, measures that are defined inconsistently with how they 
are calculated can be confusing and create the impression that 
performance is better or worse than it actually is. 29 



GAO, Internal Control: Standards for Internal Control in the Federal Government. 
GAO/AIMD-00-21.3.1. Washington, D.C.: Novemberl999. 

2 ' GAO, Executive Guide: Effectively Implementing the Government Performance and 
Results Act. GAO/GGD-96-118. Washington, D.C.: June 1996. 

28 Specifically, CBP stopped including the number of container examinations that were not 
conducted because the host government denied the request or because logistical 
challenges prevented it. As we reported in 2005, these are not necessarily indicators of 
unnecessary examinations that were prevented. 

29 GAO, Tax Administration: IRS Needs to Further Refine Its Tax Filing Season 
Performance Measures. GAO-03-143. Washington, D.C.: November 2002. 
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CBP has made efforts to enhance CSI performance measures, but we 
identified limitations in the information available for CSI program 
managers to assess the program. In the past, we and the Office of 
Management and Budget have encouraged federal departments and 
agencies to measure whether programs are achieving their intended 
outcomes, such as CSI's purpose of protecting global trade from being 
exploited by international terrorists. However, we and the Office of 
Management and Budget have acknowledged the difficulty in developing 
outcome measures for programs that aim to deter or prevent specific 
behaviors. In such an instance, we have reported that proxy measures 
should be designed to assess the effectiveness of program functions. CBP 
officials reported the agency has not been able to develop a way to 
measure the deterrence effect of the program, as CSI is designed to 
support the CBP mission to prevent and deter terrorists and terrorist 
weapons from entering the United States. Examples of CSI program 
functions include targeting and examining high-risk container shipments 
before they are loaded on vessels bound for the United States, and in our 
2005 review of CSI we provided guidance on an alternative method of 
developing proxy measures to evaluate program performance. Further, 
according to the Office of Management and Budget, proxy measures 
should be closely tied to the intended program outcome, and it may be 
necessary to have a number of proxy measures to help ensure sufficient 
safeguards are in place to account for performance results. According to 
CBP officials the following three of its existing performance measures 
were proxies for program outcomes. 30 

(1) The percentage of worldwide U.S.-bound containers passing 
through CSI seaports — since these containers are to be targeted 
and, if determined high-risk, may be examined by host government 
officials, this is a measure of the program goal to detect and 
prevent WMDs headed to U.S. seaports from leaving foreign 
seaports. 

(2) The number of foreign mitigated examinations (that is, 
examinations determined to be unnecessary due to information 
provided by host government officials and thus waived) by 



Appendix V provides more detail on the CSI performance measures (including the three 
proxies for program outcomes), the associated CSI goals, performance targets, and 
measured results. 
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category — developed to quantify whether collocating CBP officials 
at CSI seaports increases information sharing and collaboration. 

(3) The number of intelligence reports based on CSI foreign sources — 
intended to measure whether having CBP officials located at 
foreign seaports leads to increased collaboration with foreign 
customs officials. 

The Office of Management and Budget has stated that performance 
measures should capture the most important aspects of a program's 
mission and priorities. However, CBP does not have a measure that tracks 
the extent to which U.S.-bound containers carrying high-risk cargo are 
examined at CSI seaports, despite the fact that this activity is a core 
element of the CSI program. 

CBP has taken other actions to address our April 2005 recommendation 
that includes ways to improve CSI performance measures, but we found 
additional weaknesses as well. The strategic plan demonstrated how each 
performance measure corresponds to the three strategic goals of CSI, 
which include (1) securing U.S. borders, (2) building a robust CSI cargo 
security system, and (3) protecting and facilitating trade. This marked an 
improvement, as this linkage had not been made previously. In addition, 
CBP addressed an additional aspect of our prior recommendation by 
establishing performance targets for four of the six CSI performance 
measures currently used. However, only one measure had a target for 
multiple years. In addition, since issuing the CSI strategic plan the agency 
has not updated its performance targets for fiscal year 2008 or beyond for 
any of its measures. Without this information about the performance 
targets, it may be difficult for CBP to determine whether the results were 
more positive or negative than expected. 

Also, we identified a weakness in how some CSI performance measures 
are calculated. As we noted earlier in this report, as the number of CSI 
seaports has increased in recent years, program activities have increased 
as well. However, CBP does not appropriately control for this program 
growth in how it calculates three of its six performance measures. For 
example, since the "number of foreign mitigated examinations by 
category" — the number of container examinations determined to be 
unnecessary due to information provided by host government officials — is 
not calculated on a per-container basis (i.e., per 10,000 containers), it may 
be difficult to determine whether fluctuation in the numbers across years 
is due to (1) increased collaboration with foreign government officials or 
(2) simply an increase in the number of containers reviewed and 
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considered for examination at the increasing number of CSI seaports. 
Similarly, the number of intelligence reports and the number of 
investigative cases initiated may be due to an increase in the number of 
operational CSI seaports, not increased collaboration with host 
government officials. Without controlling for program growth, CBP's 
calculation of results for its performance measures may be misleading or 
confusing to CBP and DHS program managers or the Congress, who 
provide program oversight. 



Conclusions Since we began reporting on the CSI program in 2003, CBP has made 

' " significant progress in expanding and developing the program. However, 

CBP continues to face several management and operational challenges, 
which may limit CBP's ability to ensure that the CSI program provides the 
intended level of security for U.S.-bound container cargo moving through 
the international supply chain. Also, balancing security concerns with the 
need to facilitate the free flow of commerce remains an ongoing challenge 
for CBP. 



Recognizing that program evaluation data are important for program 
managers to understand why results occur and what value a program 
adds, CBP has taken actions to enhance its evaluation of CSI team 
activities. The revised evaluation program has increased the information 
available to make policy and programmatic decisions regarding the 
operations at the CSI seaports. However, limitations that remain in CBP's 
evaluation process affect the accuracy and completeness of the program 
information available for making sound management decisions about the 
CSI program as a whole. Specifically, when CBP's evaluation teams do not 
complete the evaluation tools or resolve contradictory information, 
program managers may receive limited or inaccurate information. Further, 
when the data collected using the CSI evaluation tool during the 
evaluations are not reliable and readily available for assessment, CBP's 
planned programwide trend analyses of the CSI program may be 
misleading. 

In assessing CSI performance, CBP lacks information about a very 
important aspect of the program — the overall examination systems used 
by the host governments to examine high-risk cargo shipped in containers 
as requested by CBP. CBP's efforts have led to the successful participation 
of a wide array of foreign governments in the CSI program, and CBP has 
established many cooperative relationships with its foreign partners. While 
we acknowledge the agency cannot force security requirements upon 
foreign governments, the lack of information systematically gathered 
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about the examination systems used by participating governments is 
problematic. Data about the equipment, people and processes involved in 
the examination system are vital for determining whether high-risk U.S.- 
bound containers have been properly examined or should be examined or 
reexamined upon arrival at a U.S. seaport. CBP lacks guidelines and 
criteria for most of the equipment and the people and processes used by 
host government examination systems — as required, in some instances, by 
the SAFE Port and 9/11 Acts — for evaluating CSI seaport operations and 
determining overall program effectiveness. In light of the new 9/11 Act 
requirement that 100 percent of U.S.-bound container cargo be scanned in 
the future at foreign seaports before leaving for the United States, it is 
important that that CBP have programs in place to gather the information 
necessary to ensure that cargo container examinations — and the 
equipment used as part of the examination process — are reliable, 
regardless of the point of origin. 

Program evaluations are just one source of information that managers 
need to make decisions, and evaluation data must often be coupled with 
performance measurement to assess overall program results. Measuring 
the overall impact of the CSI program remains difficult due to the 
challenges involved in creating effective performance measures, and 
because of great difficulty in measuring the deterrent effect of the 
program. As we and the Office of Management and Budget have reported, 
performance measurement can be very valuable to program managers, as 
the process can indicate what a program is accomplishing and whether 
intended results are being achieved. Measuring program performance 
encourages managers to focus on the key goals of a program and helps 
them by providing information on how resources and efforts are best 
allocated to ensure effectiveness. Though CBP identified performance 
measures it considers proxies for program outcomes (given the difficulty 
in assessing the deterrent effect of CSI), these measures do not cover a 
key core program function, for example a performance measure for the 
number of high-risk U.S.-bound containers examined at CSI seaports. 
Finally, without clearly developed performance targets for each of its 
measures, program managers, Congress, and the public lack information 
needed to determine the extent to which the CSI program is performing as 
intended. Taken as a whole, the lack of clearly articulated performance 
measures and accurate and reliable evaluative data may hinder CBP's 
ability to ensure that the resources it expends for CSI effectively achieve 
its goal of helping to secure U.S. borders against terrorists and terrorist 
weapons. 
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Recommendations for 
Executive Action 



To help ensure that CBP has the information needed to assess its 
achievement of CSI program goals to help enhance supply chain 
security — while at the same time balancing security concerns with the 
need to facilitate the free flow of commerce — we recommend that the 
Secretary of Homeland Security direct the Commissioner of U. S. Customs 
and Border Protection to take the following actions in three areas: 

• Strengthen CBP's process for evaluating CSI teams at overseas ports 
by (a) systematically capturing and maintaining all relevant evaluation 
data and documentation so that it can be used by CBP management to 
guide operating decisions, monitor program performance, and inform 
resource allocation decisions; (b) ensuring that CSI evaluation teams 
follow established evaluation procedures; and (c) monitoring the 
completion, within established time frames, of recommendations made 
in previous evaluations. 

• In collaboration with host government officials, improve the 
information gathered about the host governments' examination 
systems — which includes people, processes, and equipment — at each 
CSI port by (a) establishing general guidelines and technical criteria 
regarding the minimal capability and operating procedures for an 
examination system that can provide CBP with a basis for determining 
the reliability of examinations and related CSI activities; (b) 
systematically collecting data for that purpose; and (c) analyzing the 
data against the guidelines and technical criteria to determine what, if 
any, mitigating actions or incentives CBP should take to help ensure 
the desired level of security. 

• Enhance CSI performance measures to better assess CSI performance 
overall by (a) developing measures for all core CSI program functions 
designed to have a deterrent effect, (b) establishing annual 
performance targets — based on explicit assumptions — for all 
performance measures, and (c) revising how performance measures 
are calculated to take into account CSI program growth. 



Agency Comments 
and Our Evaluation 



We provided a draft of this report to the Department of State and the 
Department of Homeland Security for their review and comment. The 
Department of State did not provide written comments but provided 
technical comments, which have been incorporated into the report as 
appropriate. DHS provided written comments — incorporating comments 
from CBP — on December 20, 2007, which are presented in Appendix II. In 
commenting on a draft of this report, DHS noted that it concurred with 
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one recommendation and partially concurred with the remaining two 
recommendations. 

In its written comments, DHS and CBP concurred with our 
recommendation on strengthening its process for evaluating CSI teams at 
overseas locations. Specifically, CBP noted that by June 2008, it planned 
to establish a database that would contain all recommendations and action 
plans as a result of CSI port evaluations as well as due dates for 
implementing recommendations and actions taken. To ensure that CSI 
evaluation teams follow procedures, CBP indicated that it would make it 
mandatory that the teams complete all database fields. Furthermore, CBP 
reported that it would assign values to questions in its evaluation tool on 
the basis of the criticality of the activity evaluated in each question to 
CSI's mission as a whole. 

DHS commented that CBP partially concurred with our second 
recommendation to improve information gathered about host 
governments' examination systems by (a) establishing general guidelines 
and technical criteria regarding the minimal capability and operating 
procedures; (b) systematically collecting data for that purpose; and (c) 
analyzing the data against the guidelines and technical criteria. CBP 
agreed on the importance of an accepted examination process and noted it 
continues to take steps in addressing improvements in the information 
gathered about host government's examination systems at CSI seaports by 
working directly with host government counterparts, through the World 
Customs Organization, and providing capacity building training and 
technical assistance. While CBP does engage in capacity building, it does 
so with only 5 of the 33 countries with CSI ports. CBP also stated that it 
will continue to use the WCO through its SAFE Framework of Standards 
to address a uniform customs process and technical standards for 
equipment. However, the SAFE Framework mentions no specific 
technical capability criteria for inspection equipment. Additionally, CBP 
does not systematically collect or assess information on the people, 
processes, or technology used by these host governments to examine high- 
risk U.S.-bound containers. CBP also noted in its comments to this report, 
that equipment used for inspection of containers in foreign countries is 
equal to or better than the equipment used by CBP at its domestic ports. 
While CBP has performance information for the 16 seaports that have 
inspection equipment on loan from CBP, it is not in a position to assess the 
performance of equipment used at the remaining 42 CSI seaports. 
Although we repeatedly requested systematic information regarding the 
equipment technical capabilities in these other ports, CBP officials were 
unable to provide it to us. In response to our 2005 report, CBP stated that 
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it would evaluate the feasibility of technical requirements for nonintrusive 
inspection equipment, but a legal issue may exist regarding CBP's ability to 
impose such requirements. While we understand CBP's position, it could 
still gather information on such equipment's technical capabilities. 
Because the CSI inspection might be the only inspection of a container 
before it enters the United States, it is important that information on the 
people, processes, and equipment used as part of CSI be obtained and 
assessed to provide some level of assurance of the likelihood that the 
examination system could detect the presence of WMD. If a port's 
examination system were determined to be insufficient, CBP could take 
mitigating actions, such as re-examining container cargo upon its arrival at 
a domestic seaport. 

Finally, DHS commented that CBP partially concurred with our third 
recommendation to enhance CSI performance measures to better assess 
CSI performance overall. CBP stated that it believes its current measures 
address core program functions of targeting and collaboration with host 
governments to mitigate or substantiate the risk of a maritime container 
destined for the United States. We disagree. As discussed earlier in this 
report, a core element of the CSI program, specifically the extent to which 
U.S.-bound containers carrying high-risk cargo are examined at CSI 
seaports, is not addressed through CBP's performance measures. In its 
comments, CBP stated that its outcome performance indicator captures 
the number of foreign mitigated examinations by category, however CBP 
did not respond to our requests for more information regarding these 
categories, including whether risk was a category. Although it considers 
action on this recommendation completed, CBP noted its intention to 
continue to refine, evaluate, and implement measures to track progress 
toward meeting CSI objectives. As previously stated, since issuing the CSI 
strategic plan, CBP has not updated its performance targets for fiscal year 
2008 or beyond for any of its measures. Thus, we believe additional action 
is warranted. Establishing annual targets for performance measures is 
important, as agencies can determine the program's progress, in part, by 
comparing the performance measures against the targets. In addition, 
CBP did not address whether it plans to reconsider how it calculates some 
of its performance measures to control for CSI program growth. Without 
doing so, CBP's calculation of results for its performance measures may be 
misleading or confusing to CBP and DHS program managers, or the 
Congress, who provide program oversight. 

DHS and CBP also provided technical comments, which have been 
incorporated into the report as appropriate. 
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If you or your staff have any questions about this report, please contact me 
at (202) 512-9610 or at caldwells@gao.gov. Key contributors to this report 
are listed in appendix VI. This report will also be available at no charge on 
the GAO Web site at http://www.gao.gov. 




Stephen L. Caldwell 

Director, Homeland Security and Justice Issues 
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Appendix I: Objectives, Scope, and 
Methodology 



We addressed the following issues regarding the U.S. Customs and Border 
Protection's (CBP) Container Security Initiative (CSI): 

• How has CBP contributed to strategic planning for supply chain 
security efforts and the CSI program in particular, and what progress 
has been made in achieving CSI performance goals? 

• How has CBP strengthened CSI operations in response to our 2005 
review, and what challenges, if any, remain? 

• How does CBP evaluate CSI port operations and assess program 
performance overall, and how has this process changed over time? 



To address our first objective, we reviewed the strategic plans of the 
Department of Homeland Security (DHS), CBP, and CSI as well as national 
strategies like the National Maritime Security Strategy and the Strategy 
to Enhance International Supply Chain Security. We also analyzed the 
CSI strategic plan to determine whether it includes all of the key elements 
included in the Government Performance and Results Act. In addition, to 
measure CSI's progress in meeting its performance goals, we reviewed and 
analyzed CBP data related to the number of CSI seaports, the cargo CBP 
targeted and referred to the host government to examine, and the number 
of cargo containers that were (and were not) examined by host 
government officials at the CSI seaports. We also met with CBP officials 
responsible for managing the CSI program, from the CSI Strategic 
Planning and Evaluation Branch, and from CBP's Office of Field 
Operations and Office of International Affairs and Trade Relations, not 
only to gather information about CSI strategic planning and performance 
goals, but to discuss all of the issues within the scope of this review. 

To examine CBP's efforts to enhance CSI operations and the operational 
challenges that remain at CSI seaports, we reviewed GAO's previous 
assessments of the CSI program and examined CBP's efforts to implement 
our three prior recommendations. We also reviewed the CSI human capital 
plan and spoke to CBP officials about actions the agency has taken to 
ensure that CSI human resources are appropriately allocated. As part of 
that process, we met with officials at CBP headquarters and at the 
National Targeting Center - Cargo (NTCC) in Virginia to discuss the 
agency's decision to conduct some targeting of high-risk containers from 
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Appendix I: Objectives, Scope, and 
Methodology 



the NTCC rather than at CSI seaports. 1 In addition, we spoke to CBP 
officials at three domestic seaports, selected according to geographical 
location and container volume. We also visited six CSI seaports located 
overseas, and selected the locations based on geographic and strategic 
significance, container volume to the United States from the seaports, 
when the seaports began conducting CSI operations, and whether the 
seaport was involved in CBP's Secure Freight Initiative. At the CSI 
seaports, we also interviewed host government officials and CSI teams to 
discuss the frequency and level of collaboration involved in their 
interactions with each other, circumstances at seaport facilities that affect 
CSI operations, and financial cost issues associated with examinations. 
The results from our visits to seaports provided examples of CBP and host 
government operations but cannot be generalized beyond the seaports 
visited because we did not use statistical sampling techniques in selecting 
the seaports. 

To determine what progress CBP has made in strengthening its tools for 
monitoring and measuring the progress of the CSI program, we reviewed 
the performance measures presented in the CSI strategic plan against 
criteria developed by the Office of Management and Budget and GAO. In 
addition, to appraise CBP's efforts to strengthen its methods to evaluate 
CSI teams and to learn about operations at CSI seaports, we analyzed a 



1 In October of 2001, the U.S. Customs Service established the Office of Border Security 
(OBS). In October of 2002, OBS was renamed the National Targeting Center (NTC) and in 
January of 2003, was relocated to Virginia. The NTC was established in response to the 
need for proactive targeting aimed to prevent acts of terror and to seize, deter, and disrupt 
terrorists and/or implements of terror. The NTC originally combined both passenger and 
cargo targeting in one facility. As the NTC targeting mission and staff expanded, it became 
obvious that additional facilities were necessary. On March 26, 2007, NTC Passenger and 
Cargo operations were divided. The existing NTC facility became known as the NTC - 
Passenger (NTCP). The National Targeting Center - Cargo (NTCC) began operations within 
the existing facility until the NTCC facility construction was completed. On May 14, 2007, 
staff moved to the new NTCC facility located approximately a mile from NTCP. The 
mission of the NTCC is to coordinate and support CBP cargo targeting operations. NTCC 
developed enhanced operations to proactively target and coordinate examinations of high- 
risk cargo in all modes; provide high quality research and support to the Secure freight 
Initiative (SFI), Container Security Initiative (CSI), domestic CBP units, and other 
government agencies; implement new proactive methodologies and expand information 
sharing and partnerships. Liaisons stationed at NTCP include U.S. Coast Guard, Federal 
Air Marshals, Federal Bureau of Investigation, Department of State, Transportation 
Security Administration, Immigration and Customs Enforcement, and Financial Crimes 
Enforcement Network. Liaisons stationed at NTCC include the Food and Drug 
Administration, Department of Agriculture, and the Department of Commerce. The 
liaisons provide support to both facilities as needed but are physically located in one 
facility. 
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sample of evaluation documents. Our nonrepresentative sample consisted 
of evaluations for all 40 seaports for which we had documentation at the 
time of our review, including (1) the 15 evaluations conducted between 
November 2006 (when CBP revised its evaluation process and began using 
the Container Security Initiative Team Evaluation software tool) and May 

2007 (when we conducted our analysis), (2) the 7 available evaluations 
that directly preceded them chronologically and were conducted using 
CBP's previous evaluation methodology (for the purpose of comparison), 
and (3) the most recent evaluations conducted at each of the additional 
locations for which documentation had been provided by CBP. Thus, we 
reviewed a total of 34 evaluations (covering 40 CSI seaports) out of the 114 
evaluations that GAO had obtained from CBP as of May 2007. For each of 
the evaluations reviewed, we assessed any available materials, which 
could include a narrative report and/or a checklist of yes or no responses. 
While our sample covered various aspects of CBP's evaluations, our 
sample was not selected using statistical sampling techniques. Thus, the 
results from our review of CBP evaluation data provide illustrative 
examples about CSI team evaluation methods and program operations at 
CSI seaports — and generally corroborated our seaport site visit 
observations — but cannot be generalized to the all 58 seaports conducting 
CSI operations. 

We also met with CBP officials managing the CSI program to assess the 
agency's efforts to collect information about the equipment, people, and 
processes involved in the host governments' examinations of U.S.-bound 
container cargo, including the capabilities of examination equipment 
operating at CSI seaports and the proficiency of host customs 
administrations using the equipment. In addition, we selected and 
analyzed a nonrepresentative sample of 10 port assessments among those 
that CBP conducted at each port prior to its admission into the CSI 
program — the sample was composed of the 6 seaports we visited plus the 
4 highest-volume locations as of January 2007. As of that date, 
approximately 55 percent of containers bound for the United States 
passed through these 10 seaports. Thus, our findings from our review of 
the assessments provide examples about the type of information collected 
as part of the process, but cannot be generalized to all 58 seaports in the 
program. 

We conducted this performance audit from May 2006 through January 

2008 in accordance with generally accepted government auditing 
standards. Those standards require that we plan and perform the audit to 
obtain sufficient, appropriate evidence to provide a reasonable basis for 
our findings and conclusions based on our audit objectives. We believe 
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that the evidence obtained provides a reasonable basis for our findings 
and conclusions based on our audit objectives. 



Dcltcl RcliclbilitV ^ e me * W ^ °^ 1C ^ S *° discuss the agency's efforts to ensure CSI 

^ data on the number of cargo shipments and containers subject to targeting 

and examination are reliable. In our 2005 review of the program, we found 
the data to be sufficiently reliable to support our findings. Since that time, 
CBP has further enhanced the way in which it collects and aggregates 
information about CSI program activities at foreign seaports, including the 
targeting and examination of high-risk container cargo. Specifically, CSI 
teams now utilize improved technology, eliminating the need for 
transmitting data to CBP headquarters via e-mail and thereby reducing the 
opportunity for human error in manually entering and aggregating data for 
the program. CBP officials at headquarters can now directly access the 
data entered at each CSI port as soon as they are entered into the shared 
system and can monitor the data on a daily basis to identify errors in or 
mischaracterization of the data. While we did not directly test the 
reliability of 2006 data, the recent CBP initiatives to improve reliability, 
combined with GAO's previous assessment of the 2005 data, gave us 
confidence in using CSI targeting and examination data to provide 
descriptive, background information regarding the extent to which high- 
risk container cargo is targeted by CBP and examined by foreign 
governments participating in CSI. 
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U.S. Department of Homeland Security 

Washington, DC 20528 




Homeland 
Security 



December 20, 2007 



Mr. Stephen L. Caldwell 
Director, Homeland Security and Justice Issues 
U.S. Government Accountability Office 
Washington, D. C. 20548 

Dear Mr. Caldwell: 

Thank you for providing us with a copy of the draft report entitled "SUPPLY CHAIN 
SECURITY: Examinations of High-Risk Cargo at Foreign Seaports have Increased, 
but Improved Data Collection and Performance Measures are Needed" (GAO-08- 
187SU), which examines U.S. Customs and Border Protection's (CBP) Container 
Security Initiative (CSI) program and how CBP has contributed to strategic planning 
for supply chain security, strengthened CSI operations, and evaluated CSI operations 
and overall performance. 

The Department of Homeland Security (DHS) and CBP agrees with the GAO's overall 
observations and recommendation that CBP needs to enhance the data collected about 
CSI team performance. CBP is concurring in part with the recommendations made by 
GAO concerning host government examinations and the further development of 
performance measures and annual targets for core CSI functions as CBP believes it has 
addressed and will continue to work on these areas as deemed necessary. The following 
represents the Department and CBP response to the recommendations included in the 
report. 

Recommendation 1: Strengthen CBP's process for evaluating CSI teams at overseas 
ports by (a) systematically capturing and maintaining all relevant evaluation data and 
documentation so that it can be used by CBP management to guide operating decisions, 
monitor program performance, and inform resource allocation decisions; (b) ensuring that 
CSI evaluation teams follow established evaluation procedures; and (c) monitoring the 
completion, within established time frames, of recommendations made in previous 
evaluations. 

Response: CBP concurs with the recommendation and will enhance the Container 
Security Initiative Team Evaluation (CSITE) by establishing a data base that will contain 
all recommendations and action plans as a result of a CSI port evaluation. The data base 
will include due dates for each recommendation and annotate the appropriate action taken 
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and the results. In addition, the data base will be linked to CSITE in order for the 
evaluator to have a record of previous recommendations and actions taken for reference 
when conducting additional evaluations or follow-up. 

To ensure that CSI evaluation teams follow established procedures, it will become 
mandatory that all data base fields are properly completed in CSITE. This enhancement 
will address GAO's concern that not all questions in CSITE had a response. 

Moreover, CBP will establish a numerical "weight/value" for the questions in CSITE. 
Numerical "weight/value" will be set for each question and that numerical value will 
correspond to how critical a negative response would affect the CSI mission as a whole. 
With these numerical "weight/value", CBP will be able to provide an instant "Report 
Card" on the CSI port being evaluated. 

Due Date: June 30, 2008 

Recommendation 2: In collaboration with host government officials, improve the 
information gathered about host governments' examination systems - which includes 
people, process, and equipment - at each CSI port by (a) establishing general guidelines 
and technical criteria regarding the minimal capability and operating procedures for an 
examination system that can provide CBP with a basis for determining the reliability of 
examinations and related CSI activities; (b) systematically collecting data for that 
purpose; and (c) analyzing the data against the guidelines and technical criteria to 
determine what, if any, mitigating actions or incentives CBP should take to help ensure 
the desired level of security. 

Response: CBP agrees in part with this recommendation on the importance of an 
accepted examination process and continues to take steps in addressing improvements in 
the information gathered about host governments' examination systems at CSI ports by 
working directly with host government counterparts, through the World Customs 
Organization (WCO), capacity building training and technical assistance. 

CBP understands GAO's position in trying to determine the effectiveness of the foreign 
customs service conducting examinations of high-risk maritime containers destined for 
the United States. CSI has been in operation for over five years and its success is 
attributed to the cooperation and collaboration of our host government counterparts 
examining high-risk containers that are referred for inspection. 

Through the CSI program, CBP Officers work with host customs administrations to 
establish security criteria for identifying high-risk containers. With the establishment of 
security criteria, CBP has benefited in identifying high-risk containers that pose a risk for 
terrorism. Prior to CSI, many of these customs administrations were not using non- 
intrusive (Nil) technology to inspect the high-risk containers before they are shipped to 
U.S. ports. Since the inception of the CSI program, 58 CSI operational ports host 
government administrations have invested millions of dollars on Nil equipment that 
includes their purchase of radiation detection devices such as Radiation Portal Monitors 
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for use in their examination process. The level of examinations conducted at CSI 
locations increased by 93% from 70,902 in FY2006 to 136,815 in the FY2007. For the 
same time periods, the percentage of cargo examined to overall CSI related shipments 
increased by over 50% from 0.80% to 1 .25%. These increased levels of workload 
resulted in an array of enforcement actions and investigative cases. This level of success 
could not have been accomplished without the host government continued cooperation 
and having an effective examination process. 

Furthermore, host government officials have not hesitated in providing CBP with all the 
information on equipment used for the inspection of containers. This equipment is equal 
to or better than the equipment used by CBP at its domestic ports. CBP Officers are fully 
trained in the equipment being used by the host government, and in the cases where CBP 
has provided Nil equipment, those host government customs officials also have been 
trained in the use of such equipment. 

In addition to working directly with host governments, CBP provides training and 
technical assistance to customs administrations of a number of countries that currently 
participate in CSI. Such training and technical assistance forms a long-term capacity 
building program to support implementation of the WCO Framework of Standards to 
Secure and Facilitate Global Trade. The standards incorporated in the Framework 
contain key elements which support CSI including: 

« Advance electronic presentation of cargo information; 

• Screening of cargo containers using non-intrusive inspection equipment; 
% Use of automated risk management systems; 

• Standardization of targeting criteria to identify high-risk cargo and containers; 

• Employee integrity programs; and 

% Inspection of cargo in the country of origin, transit and destination. 

CBP also provides a number of assistance and training programs to foreign customs and 
border security agencies to facilitate implementation of port security antiterrorism 
measures. 

As a nation's sovereignty is critical, CBP will continue to use the WCO through its Safe 
Framework of Standards, to address a uniform customs process and technical standards 
for equipment to ensure that the examination process of cargo is one that is uniform 
throughout the world. 

Recommendation 3: Enhance CSI performance measures to better assess CSI 
performance overall by (a) developing measures for all core CSI program functions 
designed to have a deterrent effect, (b) establishing annual performance targets - based 
on explicit assumptions - for all performance measures, and (c) revising how 
performance measures are calculated to take into account CSI program growth. 

Response: CBP concurs in part with the recommendation. CBP believes that its current 
measures do address and assess CSI core program functions of targeting and 
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collaboration with the host government in order to mitigate or substantiate the risk of a 
maritime container destined to the United States. 

CBP will continue to refine, evaluate and implement any and all performance measures 
needed to track the progress in meeting any additional CSI objectives. 

CBP's current performance measures include two outcome, four information and one 
efficiency measures. These performance measures have been accepted by the Office of 
Management and Budget and are part of the DHS Performance and Accountability 
Report (PAR). 

The performance indicators include: 

• Outcome: (I) number of foreign mitigated examinations by category; and (2) 
number of investigative cases initiated due to CSI intelligence. 

• Output: (1) number of intelligence reports based on CSI foreign sources; (2) 
number of operational CSI ports; (3) number of positive findings by category; and 
(4) percentage of worldwide U.S. -destined containers processed through CSI 
ports. 

• Efficiency: average cost per CSI port to achieve operational status. 



Due Date: Completed 
Thank you again for the opportunity to provide comments to the draft report. 



Sincerely, 



Steven J. Pecinovsky v 
Director 

Departmental GAO/OIG Liaison Office 
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This appendix provides information on the 58 foreign seaports 
participating in CBP's Container Security Initiative (CSI). According to 
CBP, CSI was operating in 58 foreign seaports by the end of September 
2007. Table 2 lists the CSI seaports according to the date when the 
seaports began conducting CSI operations, shows the phase (I, II, or III) in 
which specific seaports were selected for participation in CSI, the volume 
of U.S.-bound shipments passing through the seaport in fiscal year 2007, 
and specifies which seaports are participating in the Department of 
Energy's (DOE) Megaports Initiative and in CBP's Secure Freight 
Initiative. 



Table 2: 58 CSI seaports as of September 2007 

















OcrdfJUi lo 












Volume of U.S.- 


participating in 


participating in 








CBP 




bound 


DOE's 


CBP's Secure 








selection 


Date port began 


containers, fiscal 


MegaPorts 


Freight 




Seaport 


Country 


phase 9 


CSI operations 


year 2006 


Initiative 


Initiative 11 


1 


Vancouver 


Canada 


I 


2/20/2002 


102,363 






2 


Halifax 


Canada 




3/25/2002 


26,228 






3 


Montreal 


Canada 




3/25/2002 


140,912 






4 


Rotterdam 


Netherlands 




9/2/2002 


205,461 


X 




5 


Le Havre 


France 




12/2/2002 


84,634 






6 


Bremerhaven 


Germany 




2/2/2003 


350,353 






7 


Hamburg 


Germany 




2/9/2003 


74,776 






8 


Antwerp 


Belgium 




2/23/2003 


189,466 


X 




9 


Singapore 


Singapore 




3/10/2003 


376,846 


X 


X 


10 


Yokohama 


Japan 




3/24/2003 


65,686 






11 


Hong Kong 






5/5/2003 


1,333,812 


X 


X 


12 


Gothenburg 


Sweden 




5/23/2003 


16,256 






13 


Felixstowe 


United Kingdom 




5/24/2003 


75,544 






14 


Genoa 


Italy 




6/16/2003 


104,332 






15 


La Spezia 


Italy 




6/23/2003 


130,515 






16 


Pusan 


South Korea 




8/4/2003 


610,061 


X 


X 


17 


Durban 


South Africa 




12/1/2003 


18,060 






18 


Port Klang 


Malaysia 




3/8/2004 


18,068 






19 


Tokyo 


Japan 




5/21/2004 


166,560 






20 


Piraeus 


Greece 




7/27/2004 


6,306 


X 




21 


Algeciras 


Spain 




7/30/2004 


38,266 


X 




22 


Kobe 


Japan 




8/6/2004 


87,688 






23 


Nagoya 


Japan 




8/6/2004 


75,290 







Page 58 



GAO-08-187 Supply Chain Security 



Appendix III: Container Security Initiative 
Seaports 



Seaports Seaports 





Seaport 


Country 


CBP 

selection 
phase 9 


Date port began 
CSI operations 


Volume of U.S.- 
bound 

containers, fiscal 
year 2006 


participating in 
DOE's 
MegaPorts 
Initiative 


participating in 
CBP's Secure 
Freight 
Initiative" 


24 


Laem Chabang 


Thailand 


I 


8/13/2004 


72,477 


X 




25 


Tanjung 
Pelepas 


Malaysia 


II 


8/16/2004 


24,538 






26 


Naples 


Italy 


II 


9/30/2004 


20,281 






27 


Liverpool 


United Kingdom 


II 


10/19/2004 


38,062 






28 


Thamesport 


United Kingdom 


II 


10/19/2004 


20,182 






29 


Southampton 


United Kingdom 


II 


10/19/2004 


31 ,780 


X 


X 


30 


Tilbury 


United Kingdom 


II 


10/19/2004 


5,018 






31 


Gioia Tauro 


Italy 


II 


10/29/2004 


51,664 






32 


Zeebrugge 


Belgium 


II 


10/29/2004 


13,202 


X 




33 


Livorno 


Italy 


II 


12/16/2004 


90,073 






34 


Marseille 


France 


II 


1/7/2005 


21,142 






35 


Dubai 


United Arab 
Emirates 


II 


3/26/2005 


11,316 


X 




36 


Shanghai 


China 


I 


4/12/2005 


1 ,041 ,707 






37 


Shenzhen 


China 


I 


6/24/2005 


1,099,137 






38 


Kao-hsiung 


Taiwan 


I 


7/25/2005 


780,598 






39 


Santos 


Brazil 


II 


9/21/2005 


80,146 






40 


Colombo 


Sri Lanka 


II 


9/29/2005 


97,058 


X 




41 


Buenos Aires 


Argentina 


II 


11/17/2005 


27,830 






42 


Lisbon 


Portugal 


II 


12/14/2005 


12,178 






43 


Salalah 


Oman 


III 


3/8/2006 


81 ,333 


X 


X 


44 


Puerto Cortes 


Honduras 


III 


3/25/2006 


77,707 


X 


X 


45 


Chi-lung 


Taiwan 


III 


9/25/2006 


78,150 






46 


Valencia 


Spain 


III 


9/25/2006 


64,453 






47 


Caucedo 


Dominican 
Republic 


III 


9/26/2006 


24,495 






48 


Barcelona 


Spain 


III 


9/27/2006 


46,521 






49 


Kingston 


Jamaica 


III 


9/28/2006 


54,244 


X 




50 


Freeport 


Bahamas 


III 


9/29/2006 


91,159 


X 




51 


Qasim 


Pakistan 


III 


4/30/2007 


2,058 


X 


X 


52 


Balboa 


Panama 


III 


8/27/2007 


26,543 


X 




53 


Cartagena 


Colombia 


III 


9/13/2007 


22,081 


X 




54 


Ashod 


Israel 


III 


9/17/2007 


1996 


X 




55 


Haifa 


Israel 


III 


9/25/2007 


36,594 


X 
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Seaport 


Country 


CBP 

selection 
phase 9 


Date port began 
CSI operations 


Volume of U.S.- 
bound 

containers, fiscal 
year 2006 


Seaports 
participating in 
DOE's 
MegaPorts 
Initiative 


Seaports 
participating in 
CBP's Secure 
Freight 
Initiative" 


56 


Colon 


Panama 


III 


9/28/2007 


56,098 


X 




57 


Manzanillo 


Panama 


III 


9/28/2007 


61,767 


X 




58 


Alexandria 


Egypt 


III 


9/28/2007 


4,397 


X 





Source: GAO presentation of CBP data. 



°CBP selected the foreign seaports in three phases, using the following general selection criteria for 
each phase: Phase I included the seaports with the highest volume of shipments to the United States, 
and in phases II and III additional factors were considered, such as strategic threat factors and 
diplomatic considerations. 

"As of December 2007, these seaports are in various stages of implementation of the Megaports 
Initiative, and the Department of Energy has signed agreements to begin work and is in various 
stages of implementation at seaports in additional countries as well. 

Three CSI seaports (Puerto Cortes, Honduras; Southampton, United Kingdom; and Qasim, Pakistan) 
participating in the Secure Freight Initiative will receive integrated technology that includes existing 
container scanning technology — such as X-ray and gamma ray scanners used by host nations at CSI 
ports — and radiation detection equipment. The remaining four seaports will receive more limited 
deployment of these technologies as part of the pilot program. 
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Targeting and Examining This appendix provides a detailed description of activities and equipment 
High-risk Containers used at CSI seaports to target and examine container cargo. CBP targets 

all of the U.S.-bound containers that pass through CSI seaports to identify 
and, where feasible, examine high-risk container cargo. The container 
targeting and examination activities conducted at the foreign seaports for 
U.S.-bound cargo (exports) are very similar to activities CBP conducts at 
domestic seaports for arriving containers (imports). Figure 8 illustrates the 
various steps and decision points involved in targeting and examining 
high-risk U.S.-bound containers at CSI seaports. 
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Figure 8: CSI Process for Targeting and Examining High-risk Containers Overseas 



CSI team 
receives information 
and ATS scores 
for U.S.-bound 
shipments 



Shipment 
information 

reviewed 
by CSI team 




CSI team 
conducts 
research and 
consults with 
host government 



Low risk 




Do-not-load 
order for national 
security reason 



CBP refers to 
host government 
for examination 



No 



Host government decides about examination 



Yes 



1 



No 



Request 
exam at U.S. 
port arrival 



Host government scans high-risk 
container with nonintrusive 
inspection equipment 



Host government 
conducts physical 
search 



Yes 




Host government 


^ Yes 


implements 




response plan 






Container 
loaded onto 
vessel 



Source: U.S. Customs and Border Protection. 



Under CSI, the targeting of cargo can include the targeters' review of the 
Automated Targeting System (ATS) score and the information on which it 
is based, the bills of lading — which include data about the cargo — and 
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additional information provided by host government officials. 1 CBP 
targeters at CSI seaports are to access bills of lading through ATS, a 
system that automatically uses its hundreds of rules to check available 
data for every container arriving in the United States and assigns a risk 
score to each cargo shipment. 2 Targeters review the bill of lading, making 
a cursory check for discrepancies and anomalies in the name and address 
of the importer, the commodity, the cargo description and other data 
elements. On the basis of the initial review of the bill of lading, CBP 
officials are to either (1) categorize the cargo as low risk, in which case, 
the container holding the cargo is loaded onto the departing vessel without 
being examined, or (2) conduct further research in order to properly 
characterize the risk level of the cargo. Further research entails targeters 
using automated resources, such as the Treasury Enforcement 
Communication System or AutoTrack, as well as nonautomated resources, 
such as information provided by host government officials, to obtain 
applicable information to determine the validity of the shipment. Further 
research may also be conducted by the team's intelligence research 
specialist. After further research is completed, CBP officials are to 
characterize the cargo as either (1) low risk, in which case it is loaded 
onto the departing vessel without being examined, or (2) high-risk, in 
which case it is referred to host government officials for concurrence to 
examine. Since CBP officials do not have the legal authority to examine 
U.S.-bound containers in foreign seaports, the host government customs 
officials conduct the examinations. 

Host government officials can respond to the referrals for examination in 
one of three ways — cargo is examined or the request is either waived or 
denied. After receiving a referral from CSI teams, host customs officials 
are to review the bill of lading and the reasons for the referrals to 
determine whether or not to examine the container cargo. Some host 
governments collect intelligence information on U.S.-bound cargo 
independent of CSI, which host officials also consider in decisions of 
whether to examine the referred cargo. If host government officials agree 
that the cargo is high-risk, they will proceed with an examination. 
According to CBP, in general, CSI team members are to observe the 
examinations and review and document the results. On the basis of the 
results of a nonintrusive examination, such as if an anomaly is apparent in 



1 Bills of lading are documents issued by a carrier describing the goods, the details of the 
intended voyage, and the conditions of transportation. 

2 A risk score is derived from applicable rules that assess the level of risk for the shipment. 
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the image of the container, the host government and CBP officials must 
decide whether the host government will conduct a physical examination 
of the a container. 

Alternatively, the CSI team may waive an examination referral if (1) host 
government officials provide the CSI team with additional information that 
lowers the risk level of the cargo or (2) logistics prohibit an examination, 
such as if the cargo container were already loaded on the departing vessel. 
Finally, if the host government officials determine, on the basis of their 
review, that the cargo is not high-risk, they will deny examination of the 
cargo. For any high-risk cargo for which an examination is waived or 
denied, CSI teams are to place a domestic hold on the cargo, so that an 
examination will be conducted upon arriving in the United States. 
However, if CSI team members are adamant that a cargo container poses 
an imminent risk to the carrier or U.S. seaport of arrival but cannot 
otherwise convince the host officials to examine the container, CSI team 
members are to contact and coordinate with the NTCC to issue a do-not- 
load order for national security. According to CBP officials, this order 
advises the carrier that the specified container will not be permitted to be 
unloaded in the United States until a time when any associated imminent 
risk to the cargo container is neutralized. Once the risk is neutralized, the 
container is to be loaded back onto the carrier and placed on hold for a 
domestic examination. According to CBP officials, this type of do-not-load 
order for national security has been implemented six times since the 
inception of CSI. 



There are generally two types of CSI cargo container examinations — 
scanning with nonintrusive inspection equipment and physical searches. 
For scanning cargo containers, there are two basic types of nonintrusive 
inspection equipment currently used at CSI seaports: (1) radiation 
detection equipment and (2) imaging inspection equipment, which may 
use X-rays or gamma rays. Radiation detection equipment, such as a 
radiation portal monitor (RPM) and radiation isotope identifier devices 
(RIID) detects the presence of radioactive material that may originate 
from a container. However, only the RIID can determine whether the type 
of radiation emitted by the material actually poses a threat or is a normal 
emission of radiation, such as that found in ceramic tile. We observed at a 
domestic and a foreign seaport that generally if radioactive emissions are 
detected from a cargo container, customs officials will use a RIID (shown 
in fig. 9), to determine whether the radiation being emitted poses a threat. 



Equipment Used to 
Conduct Examinations of 
Cargo Containers 
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Figure 9: CBP Official Using Radiation Isotope Identifier Device to Examine 
Container at CSI Seaport 




Source: GAO. 



The second type of equipment, referred to as imaging equipment, uses X- 
ray or gamma ray technology to scan a container and create images of the 
container's contents without opening the container. CBP officials, along 
with host government officials, may review the images produced with the 
X-ray or gamma ray equipment to detect anomalies that may indicate the 
presence of WMD. Figure 10 shows a sample image produced by this type 
of equipment. 
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Figure 10: Commercial Sample Image Produced by Nonintrusive Imaging X-ray 
Equipment of a Container Loaded on a Truck Trailer 




Source: Host government customs organizations. 



The capabilities of nonintrusive imaging inspection equipment vary by 
manufacturer and model. In May 2005, CBP defined minimum 
performance capabilities to evaluate the quality and performance of the 
nonintrusive imaging inspection equipment being considered for use at 
domestic seaports. The domestic standards set baseline performance 
requirements for such things as the ability of nonintrusive inspection 
equipment to identify images through steel shielding (referred to as 
penetration) or the ability to scan an amount of containers in a given time 
(referred to as throughput). 
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This appendix provides information on the CSI performance measures 
used by CBP. Table 3 describes the performance measures CBP is 
currently using to report the overall performance of the CSI program, the 
linkage between performance measures and CSI goals, the performance 
targets established, and the recent results collected for each measure. In 
addition, since our 2005 report, CBP has identified performance measures 
one, two and three below as its proxy measures used in place of a measure 
for program outcomes, given the difficulty in measuring the deterrent 
effect of the program. 



Table 3: CSI Performance Measures 



Performance 

measure Performance 
number 9 measures 



Description of performance 
measures 



CSI goals associated 
with specific 
performance 
measures 



Performance targets and actual 
results of measures, by fiscal year 



Targets 



Results 



1 c 


Percentage of 
worldwide United 
States destined 
containers 
processed through 
CSI seaports" 


Percentage of total U.S.- 
bound containers that pass 
through CSI seaports, where 
actions are taken to detect 
and prevent WMDs from 
leaving the foreign seaports. 


Goal 1 : securing U.S. 
borders 


2010: 
86% 


2007: 
86% 


2 = 


Number of foreign 


Number of container 


Goal 1 : securing U.S. 


2006: 


2006: 



mitigated 
examinations by 
category 



examinations determined to 
be unnecessary, and thus 
waived, due to information 
provided by foreign customs 
administrations. Developed to 
quantify whether co-locating 
CBP officials at CSI seaports 
increases information sharing 
and collaboration. 



borders 

Goal 2: building a 
robust CSI cargo 
security system 



24,000 



No target 
established. 



47,630 



2007: 
18,404 



Number of 
intelligence reports 
based on CSI 
foreign sources 



Number of intelligence reports 
that CSI teams disseminate to 
other CSI and CBP offices 
after researching information 
provided by CSI foreign 
sources. Intended to measure 
whether having CBP officials 
located at foreign seaports 
leads to increased 
collaboration with foreign 
customs officials. 



Goal 1: securing U.S. 
borders 

Goal 2: building a 
robust CSI cargo 
security system 

Goal 3: protecting and 
facilitating trade 



No target 
established. 



2007: 
138 
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Performance 

measure Performance 
number 9 measures 



Description of performance 
measures 



CSI goals associated 
with specific 
performance 
measures 



Performance targets and actual 
results of measures, by fiscal year 



Targets 



Results 



Number of 
operational CSI 
seaports 



Total number of seaports 
where CSI has been 
implemented and is operating. 



Goal 2: building a 
robust CSI cargo 
security system 

Goal 3: protecting and 
facilitating trade 



2006: 
50 



2006: 
50 



2007: 
58 



2007: 
58 



Number of 
investigative cases 
initiated due to CSI 
intelligence 



Number of investigative cases Goal 1 : securing U.S 
opened either in the United borders 
States or at a foreign location 
due to intelligence gathered 
by CSI staff at foreign port 
locations. 



2006: 
48 



2006: 
155 



Goal 3: protecting and 
facilitating trade 



2007: 

No target 
established. 



2007: 
135 



Number of positive 
findings by category 



Number of foreign and 
domestic positive findings in 
U.S. -bound containers in 
which there was participation 
from CSI teams. 



Goal 1 : securing U.S. 
borders 

Goal 3: protecting and 
facilitating trade 



No target 
established. 



2007: 




Source: GAO presentation of data reported by CBP. 

"The CSI strategic plan includes seven performance measures, but according to CBP officials, the 
agency is no longer using its cost efficiency measure, which tracked the expense involved in 
integrating a new foreign seaport into the CSI program. CBP officials stated that they did not plan to 
establish a cost efficiency measure for ongoing CSI seaport operations because the seaports differed 
greatly in their costs. 

"CBP defines 'processing' to include any of the following CSI activities: screening U.S. destined cargo 
using ATS; conducting further research; collaborating with host government officials; or examining the 
container. 

These three performance measures are those CBP identified as proxies for program outcomes. 
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and provides analyses, recommendations, and other assistance to help 
Congress make informed oversight, policy, and funding decisions. GAO's 
commitment to good government is reflected in its core values of 
accountability, integrity, and reliability. 

The fastest and easiest way to obtain copies of GAO documents at no cost 
is through GAO's Web site (www.gao.gov). Each weekday, GAO posts 
newly released reports, testimony, and correspondence on its Web site. To 
have GAO e-mail you a list of newly posted products every afternoon, go 
to www.gao.gov and select "E-mail Updates." 



Order by Mail or Phone 



The first copy of each printed report is free. Additional copies are $2 each. 
A check or money order should be made out to the Superintendent of 
Documents. GAO also accepts VISA and Mastercard. Orders for 100 or 
more copies mailed to a single address are discounted 25 percent. Orders 
should be sent to: 

U.S. Government Accountability Office 
441 G Street NW, Room LM 
Washington, DC 20548 



To order by Phone: 



Voice: (202) 512-6000 
TDD: (202) 512-2537 
Fax: (202) 512-6061 



To Report Fraud, 
Waste, and Abuse in 
Federal Programs 

Congressional 
Relations 

Public Affairs 



Contact: 

Web site: www.gao.gov/fraudnet/fraudnet.htm 
E-mail: fraudnet@gao.gov 

Automated answering system: (800) 424-5454 or (202) 512-7470 

Gloria Jarmon, Managing Director, jarmong@gao.gov, (202) 512-4400 
U.S. Government Accountability Office, 441 G Street NW, Room 7125 
Washington, DC 20548 

Chuck Young, Managing Director, youngcl@gao.gov, (202) 512-4800 
U.S. Government Accountability Office, 441 G Street NW, Room 7149 
Washington, DC 20548 
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